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The purpose of this study was to explore the engineering problem finding ability of high 
school students at three high schools in Minnesota. Students at each of the three schools 
had differing backgrounds including pre-engineering coursework, traditional technology 
education coursework and advanced science coursework. Students were asked to find 
problems in two different engineering scenarios which were presented to them on a 
paper and pencil instrument. Responses were scored by a panel of judges based on 
measures of creativity (flexibility, fluency, originality and elaborateness) and analyzed 
based on demographic data including gender, prior coursework and school. In addition 
student responses were categorized and evaluated qualitatively based on school and 
gender of respondent. Quantitative results indicate that the most consistent predictor of 
creativity in engineering problem finding scenarios was the number of advanced science 
classes. Specific measures of creativity included other significant predictors but 
advanced science coursework was the most consistent across all measures and 
scenarios. The qualitative results showed striking differences in the responses from 
students at different schools. Students from schools with a pre-engineering and 
advanced science emphasis found similar categories of problems and had a similar view 
of the purview of engineers while students with a technology education background 
focused on a rather different set of problems and had a much narrower view of 
engineering. Results show clear differences in the types of problems found by students 
at these three high schools as well as their understanding of the scope of engineering 
problems. Educators need to become more aware of the importance of problem finding 



in engineering and better encourage the development of problem finding skills among 




their students. Specifically, technology education teachers may need supplemental 
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professional development related to the scope of engineering and engineering problem 
finding as well as how these concepts might be infused into their curriculum and 
encouraged among their students. 
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The Problem 

Our lives and world seem to be teeming with problems. From how to save on 
gasoline to how to keep food from spoiling; and from how to maintain our ability to 
feed a growing population to how to protect endangered species we live in a world 
which seems defined by problems. For all of recorded history we as a society have 
relied on science and technology to help address problems such as these. From the 
earliest methods for farming to the invention of the wheel, sail and ship to the more 
recent advances brought about by electronics, computers and the Internet we see the 
importance of technology as a method for addressing a growing number of problems. 
Still, all of these advances in science and technology would remain purely intellectual 
curiosities were it not for people who can see a practical value to them and who have an 
ability to apply them to solving real problems. 

As with most useful and productive services, over time a profession of people 
who use science and technology to solve real world problems has emerged. Many of 
those involved in this type of problem solving are identified either by themselves or 
others as engineers. The U.S. Department of Labor describes an engineer as someone 
who applies the principles of science and mathematics to economically solve technical 
problems (U.S. Department of Labor, 2007). Thus, when we talk about solving real 
world problems using technology and scientific discoveries in a systematic way we are 
really talking about engineering problem solving. It naturally follows, then, that 
engineers who are most familiar and most successful with problem solving and the 
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skills it requires will turn out the highest quality solutions to the problems they 
examine. 

As we do not live in an ideal world it is not enough though for engineers to 
design the best possible solution to a problem. Even in the definition provided by the 
Department of Labor we can see the influence of commercialization and economics. 

The solutions devised by engineers must not only be technically and scientifically 
feasible but must also be economically feasible. This wrinkle in engineering problem 
solving, which requires that a solution be economical, can add quite a bit of complexity 
to the problem solving process and necessitates that the best and most desirable 
engineers have superior problem solving skills. 

Although the success of engineers is largely determined by their ability to solve 
these complex and ill-structured “real world” problems engineering schools often do a 
poor job of teaching students how to deal with these types of problems (Jonassen, 
Strobel, & Lee, 2006). The solving of problems with unclear goals and unlisted 
constraints is not something that students are able to learn through the structured 
problem solving they are most commonly introduced to in engineering classes 
(Jonassen et ah, 2006). The problems discussed in school are typically designed to have 
a correct or at least apparently optimal solution to which the student’s solution can be 
compared. In the real world problems often have more than one appropriate solution 
and can require the balance of conflicting goals and other complex issues. More recent 
research has suggested that even the word or story problems that engineering students 
might be exposed to do not appear to provide adequate transfer to the solving of 



complex, ill-structured problems (Jonassen et ah, 2006). 
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It is not just engineers though, who benefit from the practical application of 
science and technology to solve problems. As our daily lives have evolved and become 
more complex we have exposed ourselves to an increasing number of technological 
problems. For example, the majority of Americans living today take for granted the 
ability to control the climate in their homes through heating and cooling systems. The 
function of a building has evolved from a simple shelter from the elements to something 
expected to keep us comfortable year round but the increasing cost of energy is slowly 
causing us to re-think our living habits. Thus, the homeowner must now balance the 
desire for a comfortable home with the reality of the costs for maintaining such a home. 
These seemingly contradictory goals might be achievable or the situation improved 
though the application of scientific and technological principles in a problem solving 
process and the better that process the more satisfactory the end result. 

Problems 

It may not seem necessary to define what is meant by a problem. After all, we 
frequently use the term and nearly everyone has experienced a problem of some kind 
but when embarking upon a study of problems it soon becomes evident that “problem” 
is not a universal term and can have multiple meanings which must be differentiated. 

Psychologist Karl Duncker (1945) begins his work on problem solving by 
describing problems as what occurs when someone has a goal but does not yet know 
how to meet that goal. This description of a problem as something which blocks the 
move from an existing state to a desired state is echoed by Simon & Newell (1972). 
Along these lines it is possible to describe what blocks this transition as a difficulty. The 



Oxford English Dictionary describes the most common uses of problem to be “a 
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difficult or demanding situation; a matter or situation regarded as unwelcome, harmful 
or wrong and needing to be overcome; a difficulty” (2008) a description which clearly 
fits those provided by Duncker and Simon & Newell. 

It is not merely enough though to describe a problem as a difficulty. Those 
problems faced by engineers and designers, while they may be difficult, are more than a 
difficulty. Merriam-Webster’s Dictionary defines a problem as “a question raised for 
inquiry, consideration, or solution” (2008) which is a more appropriate definition in this 
case. Getzels (1982) suggests that in these cases instead of being a difficulty the 
development of the problem is itself the primary goal and what remains then, is 
execution. Furthermore, it becomes possible to classify problems based on whether the 
problem exists or is created, whether the problem is suggested by the solver or another 
and whether a known solution exists or must be devised. This method for classification 
of problems led Getzels (1964; 1982) to describe ten common types of problems: 

1 . The problem is given (is known) and there is a standard method for solving 
it, known to the problem solver (student, experimental subject) and to others 
(teacher, experimenter) and guaranteeing a solution in a finite number of 
steps. 

2. The problem is given (is known) but no standard method for solving it is 
known to the problem- solver, although known to others. 

3. The problem is given (is known) but no standard method for solving it is 
known to the problem-solver or others. 

4. The problem itself exists but remains to be identified or discovered (become 



known) by the problem solver, although known to others. 
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5. The problem itself exists but remains to be identified or discovered (become 
known) by the problem solver and by others. 

6. The problem itself exists but remains to be identified or discovered (as in 4 
and 5) and there is a standard method for solving it, once the problem is 
discovered known to the problem solver and to the others (as in 1). 

7. The problem itself exists but remains to be identified or discovered and no 
standard method for solving it is known to the problem solver, although 
known to others (as in 2). 

8. The problem itself exists but remains to be identified or discovered, and no 
standard method for solving it is known to the problem solver or to the 
others (as in 3). 

9. The problem does not yet exist but is invented or conceived, and a method 
for solving it is known or becomes known once the problem is formulated. 

10. The problem does not yet exist but is invented or conceived, and a method 
for solving it is not known. 

Jonassen (2000) takes a slightly different tack and suggests two critical 
attributes that something must have if it is to be a problem. First, there must be a 
situation with an unknown which is described as a discrepancy between a current state a 
goal state. This is similar to the definition used by Simon & Newell (1972). Secondly, 
there must be some social, cultural or intellectual value to finding or solving the 
unknown. The value could be either intrinsic or extrinsic but the key component is that 
someone feels that it is worth finding the unknown. Jonassen (2000; 2006) believes that 
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there are three essential types of problems which form the basis for problem 
classification. 

Puzzle problems are characterized by having a single correct solution which is 
arrived at using a specific procedure (Jonassen, 1997). Although multiple methods may 
accomplish the same end result only the single most efficient method is deemed to be 
the correct one. These are problems which have most commonly been associated with 
the study of cognitive problem solving, for example by Simon & Newell (1972). Such 
problems include the Tower of Hanoi, water jug problems and the nine dot problem 
familiar to cognitive process researchers. Jonassen (1997) suggests that while these are 
interesting from an initial research perspective they map poorly onto complex real- 
world problems and, as such, are not relevant to school learning or everyday practice. 

Well-defined or well- structured problems are those which people are most 
familiar with and conditioned to solve, especially in the school setting. For example, 
when a math or science teacher writes problems for an exam or assigns “homework 
problems” they are most frequently well-defined problems. This is likely because well- 
defined problems have a definite solution process which requires the application of 
concepts, rules and principles from a given knowledge domain (Jonassen, 1997). In 
other words, well-structured problems are good for checking basic understanding and 
facts, something which is often the desired outcome of homework and exams. Jonassen 
(1997; 2000) describes well- structured problems as having a well-known initial state, a 
defined goal and known method for arriving at a solution. Although not explicitly 
identified as well- structured the first and second type of problems identified by Getzels 



(1964; 1982) are really examples of well-defined problems. These are given problems 
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with standard methods for solving the problem and solutions can be compared to 
determine correctness. One common misconception about well-defined problems which 
Jonassen (1997) wishes to dispatch is the idea that skills learned in solving them will 
easily transfer to real world, ill- structured problems. 

The third category of problem is the ill-structured or ill-defined problem. Unlike 
puzzle problems and well-structured problems this category of problem is frequently 
tied to a specific context and the information required to solve the problem is not all 
available in the problem statement (Jonassen, 1997; Jonassen, 2000). Many, if not most, 
of the problems encountered in daily life are of the ill-structured variety (Jonassen, 

1997; Hill & Smith, 1998). For example, problems found or given to engineers and 
designers do not often contain all of the information required to solve the problem in the 
problem statement. There is also significant room for individual creativity in these types 
of problems as more than one correct solution may exist. Ill- structured problems will 
often require the application of multiple domains of knowledge and judgment calls 
which must be made by the problem solver (Jonassen, 1997). The problem types 
Getzels (1964; 1982) identifies as three through ten could be described as ill-structured 
problems in that multiple solutions might exist and the problem and/or the method for 
solving it is not fully understood. 

For the purposes of this study the problems of interest are ill-defined questions 
raised for inquiry, consideration or solution. This is undoubtedly the most frequently 
encountered type of problem in the real world and that which is faced by designers and 
engineers on a daily basis. Although designers and engineers are often asked to “fix a 



problem” or are given what initially appear to be problems, they are often more 
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accurately described as dilemmas, issues or scenarios. There is usually no immediately 
apparent path to solution and devising a solution will require the applieation of critical 
thinking skills and creativity. The dilemma or issue must also be somehow turned into a 
problem which can be solved. 

Problem posing 

The study of problem solving and the desire to utilize it in schools and 
workplaces is not a new idea. Early twentieth eentury educational researcher John 
Dewey (1910) proposed a five-step problem solving proeess which included: (1) felt 
difficulty, (2) problem elarification, (3) identification of possible solutions, (4) testing 
of solutions, and (5) verification of results. The four step heuristie model for problem 
solving proposed by mathematics researcher George Polya (1957) is similar to that of 
Dewey as it includes (1) understanding the problem, (2) devising a plan, (3) earrying 
out the plan, and (4) looking back, all of which can be seen in Dewey’s five steps. One 
problem with models such as these are that they make the eomplex task of problem 
solving appear to be deceptively simple and can hide some of the processes whieh result 
in the most desirable solution. In his seminal study of problem solving psychologist 
Karl Duncker (1945) suggests that the process of finding a solution is more accurately 
seen as the eontinual reformulation of the problem. Over time this problem 
reformulation leads to the discovery of “essential” properties of the solution whieh, 
given knowledge of the domain, will in turn dictate an appropriate solution to the 
problem. 

It may initially seem like a radical suggestion that the formulation of the 
problem is more essential than the solution but Duneker is not alone in this belief. 
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Scientists Albert Einstein and Leopold Infeld noted the importance of problem 
formulation in their discussion of the evolution of physics: 

The formulation of a problem is often more essential than its solution, which 
may be merely a matter of mathematical or experimental skill. To raise new 
questions, new possibilities, to regard old problems from a new angle, requires 
creative imagination and marks real advance in science. (Einstein & Infeld, 

1961, p. 92) 

Einstein illustrates the importance of problem formulation by discussing the problem of 
determining whether light travels instantaneously or whether it occupies time, as sound 
does, a question posed by Galileo. Although the crude instruments of Galileo’s time 
prohibited him from answering this question once he had formulated the question he 
was able to discern an experimental procedure which could be used, leaving the work to 
be done a matter of technical and experimental skill (Einstein & Infeld, 1961). Clearly, 
the experiment itself is not the most difficult part of this problem. Instead, the 
formulation of the problem is the true challenge and if we are to be good at the solving 
of problems we must be good at the finding of problems. 

It is sometimes the case that when the topic of problem finding comes up it is 
asked “why do we need to seek out more problems when our world is already full of 
them?” While it is certainly the case that our world has many issues, dilemmas and 
quagmires but for these there is no immediately apparent path to solution (Getzels, 
1979), they are not solvable in their current form. Returning to our earlier discussion of 



problems this can additionally complicate the categorization of problems as the initial 
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dilemma may be presented to the problem solver but the problem to be solved remains 
to be found. 

Take, for example, early prairie farmers who were able to harvest an abundance 
of grain on their remote but fertile ground. Someone may consider the problem to be 
that we did not have enough roads to transport grain to markets and ponder how to get 
roads built in these remote areas, but taking this view the problem domain has already 
been narrowed to building roads. As it turns out roads are not the only way to transport 
grain and may not be the most efficient for a specific situation. If a river is nearby or a 
number of farmers deposit grain in an elevator near them to which a railroad could be 
built these may be better solutions. Solving the problem for these farmers is aided by 
the finding or formulation of an underlying problem, the transportation of grain from 
farm to market, and the posing of questions such as the feasibility of centrally collecting 
grain prior to shipment or the proximity of a navigable river which may provide a better 
solution. 

If then, the ability to solve complex and ill-structured problems is a critical skill 
and the formulation of problems or problem posing a critical and early step in the 
process of solving these types of problems should we not focus attention upon problem 
finding as a desired outcome of technology and engineering education? Unfortunately, 
problem finding has been largely neglected by researchers both as a whole (Getzels, 
1979) and in the field of technology education (Lewis, Petrina, & Hill, 1998). It is the 
goal of this study to identify the state of problem finding in the high school classroom 
as it relates to the solving of technological problems and to compare the problem 
finding abilities of students from a wide variety of backgrounds. 
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Problem solving 

Once a problem has been found the business of devising a solution to the 
problem can begin. The most basic description of problem solving comes from the 
seminal work of Newell & Simon (1972) which describes problem solving as the 
transition from an initial situation to a goal state by a narrowing of the problem space. 
The initial state, sometimes referred to as the problem state, encompasses the 
understanding of the situation as it exists. Take for example the dilemma of energy 
efficiency given earlier. In this case the problem state could include the thickness of 
walls, amount of insulation, price of energy, type of heating and cooling system, area 
climate and many other such factors. The goal state is the desired result and embodies 
the solution to the problem. In our example of home energy efficiency this could be the 
replacement of the heating system, additional insulation, use of passive solar design or 
many other solutions all of which could reduce climate control costs while improving 
efficiency. The link between the problem state and the goal state is the search for 
solutions through the narrowing of the problem space. This narrowing occurs as the 
problem solver searches through all the information they have access to which seems 
relevant to the problem including things in their memory and any research they conduct. 
Eventually the problem solver narrows the problem space enough to determine the 
solution to the problem (Newell & Simon, 1972). 

Even in the example of home energy efficiency we can begin to see deficiencies 
in Newell & Simon’s description of problem solving. The problem has more than one 
solution and the problem space cannot be systematically narrowed until a solitary 



solution becomes obvious. Complex and ill-defined problems such as those most 
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commonly faced by engineers and designers cannot be solved in Newell & Simon’s 
simple problem space model (Middleton, 2005). 

Recognizing the deficiencies in the simple problem spaee model as well as the 
extensive use of design in Australian technology classrooms Middleton (2005) modified 
Newell & Simon’s simple problem space model to aceount for the characteristics of 
design problems ineluding the ill- structured nature and potential existenee of multiple 
solutions. The revised model replaces the problem state with a “problem zone” which 
allows you to start with an ill-defined and eomplex problem about whieh little is known. 
The goal state is replaced by a “satisficing zone” indieating an understanding that more 
than one eompeting goal may exist and a balanee may need to be struck as well as the 
understanding that multiple solutions may exist. Finally, the simple narrowing of the 
problem space by searching is replaced by a complex seareh and eonstruction process 
where numerous procedures are used whieh may be eonstructed or emerge. Another 
critical aspect of the revised model is that it is not a linear one, there remains a back and 
forth between the problem zone and the seareh and construction space as well as 
between the satisficing zone and the seareh and eonstruction space (Middleton, 2005). 
This baek and forth allows the problem to be redefined and the solution reexamined as 
the problem solving continues. 

A focus on design and engineering 

Technology education in the United States, like industrial arts and manual 
training before it, has traditionally foeused on domain knowledge and production skills 
rather than on intelleetual processes. In 1983 the landmark government report A Nation 



at Risk was published. Among other indicators that the American system of public 
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education was on a dangerous path it was reported that high school students were 
lacking in intellectual skills, such as problem solving, which should be expected of 
them (The National Commission on Excellence in Education, 1983). The concerns 
identified in A Nation at Risk were reiterated in 1991 with the publication of What Work 
Requires of Schools: A SCANS Report for America 2000. Again, intellectual skills and 
problem solving were identified as lacking as critical skills in American students (The 
Secretary's Commission on Achieving Necessary Skills, 1991). Taking note of this shift 
in the workplace, Johnson (1992) wrote in the Journal of Technology Education about 
the critical nature of intellectual skills and suggested that the field adopt a curriculum 
emphasizing intellectual skills such as problem solving through ill-structured, design 
oriented problems. 

In 2000 the International Technology Education Association published the 
Standards for Technological Literacy, a set of content standards for the study of 
technology in schools. Out of the twenty standards four are entirely focused on the 
design process and several others make note of the importance of design (International 
Technology Education Association, 2000). Many cognitive scientists consider design to 
be a special case of problem solving (Newell & Simon, 1972) so the inclusion of design 
in standards such as these can be interpreted as an explicit inclusion of problem solving. 

Although technology educators in the United States have been slow to fully 
integrate a design process in their curriculum other parts of the world have truly 
embraced design as a foundation for technology education. The British, too, were 



concerned about the gap between education and industry but their wake up call came 
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with the release of the Crowther Report in 1959 (Gradwell, 1996), several decades 
before A Nation at Risk and the SCANS report were released in the United States. 

Beginning in 1963 British researcher Gerd Sommerhoff opened the Technical 
Activities Centre at Sevenoakes School in Kent (Gradwell, 1996). This center was 
designed squarely to address the gap between education and the workplace, especially 
the field of engineering, through student designed projects requiring both creativity and 
problem solving skills (Gradwell, 1996). From the late 1960s through the 1970s the 
ideas promoted by Sommerhoff were propagated to other schools in Great Britain and 
design based problem solving slowly became a regular part of the curriculum 
(Gradwell, 1996). The late 1980s and early 1990s saw the introduction of a national 
curriculum in the United Kingdom aimed at ensuring that all students learn essential 
knowledge and skills. One of the subjects selected for inclusion in this curriculum was 
Design & Technology, a subject heavily influenced by the ideas of Sommerhoff almost 
thirty years earlier (Atkinson, 1990; Gradwell, 1996). 

The advent and subsequent revisions of the national curriculum in the United 
Kingdom have firmly entrenched the idea of design and problem solving within their 
technology curriculum while schools in the United States continue to struggle with the 
idea, though some progress is being made. In recent years the field of technology 
education in the United States has been moving towards a more engineering and design 
focused curriculum. Articles by respected technology education researchers (Lewis, 
2005a; Wicklein, 2006) as well as the inclusion of design standards in the ITEA 
Standards for Technological Literacy (2000) indicate the increasing interest in 



engineering and design as content and a framework for the study of technology. As 
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engineering and design curriculum continue to propagate through American schools 
there will need to be continued research which aids us in understanding student problem 
solving both in how problem solving can be nurtured in students as well as appropriate 
methods for the assessment of problem solving. 

The study presented here focuses on describing the state of engineering problem 
finding at three high schools in Minnesota. Each of the schools is unique in how 
students come to know and understand engineering. One school follows the Project 
Lead the Way engineering program, a second has traditional technology education 
courses and a third which has neither but prides itself in having extensive enriched and 
advanced science courses. The student bodies are also different with one school located 
in an exurb, the second in a rural town of moderate size and the third in a first ring 
suburb which prides itself in high achievement. 

Students at these three high schools were asked to take part in this anonymous 
study of engineering problem finding by providing basic demographic information 
including gender and a list of high school engineering, technology and advanced 
science classes. The students were then presented with two engineering scenarios and 
asked to develop and describe problems which, if solved, would improve each situation. 
The first of the two scenarios revolved around increasing the energy efficiency of 
homes under development while the second asked about a town subject to flooding 
issues from a local river. A copy of the instrument used is located in Appendix A. 
Problem Statement 

Complex and ill-structured problems such as those commonly found within the 
design and engineering fields require that solvers have the ability to define and question 
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the problem itself through a process of problem finding but the research on problem 
finding in technology and engineering education has not been extensive and we do not 
know whether high school engineering curricula support this critical skill in problem 
solving. 

Guiding Research Questions 

1 . How do the problems identified by students at three Minnesota high schools 
compare on known measures of creativity? 

2. Does gender make a difference in the types or creativity of problems posed by 
students from three Minnesota high schools? 

3. Does coursework or curriculum make a difference in the types or creativity of 
problems posed by students from three Minnesota high schools? 

4. What do the problems posed by students at three Minnesota high schools reveal 
about their understanding of the nature of engineering? 




Chapter II 
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Review of the Literature 

As has already been indicated in the first chapter there has been a recent turn 
towards engineering and design in the field of technology education. The publishing of 
the Standards for Technological Literacy (International Technology Education 
Association, 2000) with their emphasis on design makes it a foregone conclusion that 
design will continue to increase in importance within the American technology 
education curriculum as time goes on. Arguing that the field should go one step further 
and specify engineering design as the primary content for the study of technology are 
researchers such as Wicklein (2006) and Bensen & Bensen (1993). When combined 
with movement in the engineering field emphasizing the importance of technological 
literacy by groups such as the Institute of Electrical and Electronics Engineers (IEEE) 
though their pre-university education division and the desire of the National Academy 
of Engineering to make engineering more accessible to students from a variety of 
backgrounds (Pearson & Young, 2002) it seems that engineering design will become a 
part of technology education. 

The types of design based problems which an engineering design curriculum 
suggests will be somewhat different in nature than the assignments which have been 
typical in the technology education and industrial arts classrooms of the past. Eor one 
thing some researchers have called for the integration of more math and science in 
engineering design (Cotton, 2002; Roman, 2001; Wicklein, 2006). Other researchers 
such as Eewis (2005a) have noted that students and teachers will need to have content 
knowledge of engineering design as well. 
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Engineering design and problem solving 

Although there is not one unified engineering design method or process across 
all fields or schools of engineering there is some agreement on the basic stages of 
engineering design. Furthermore, if we consider design to be a specialized form of 
problem solving, as was indicated in chapter one, and we consider engineering design to 
be a subset of design we can begin to see the interaction between problem solving 
methods and engineering design methods. Take for example the five step problem 
solving method proposed by Dewey (1910): 

1. Felt difficulty 

2. Problem clarification 

3. Identification of possible solutions 

4. Testing of solutions 

5. Verification of results 

Compare this with the thirteen step engineering design process used in the engineering 
school at Dartmouth (Garmire, 2003): 

1. Define the problem 

2. Restate the problem 

3. Develop constraints/criteria/specifications 

4. Brainstorm ideas 

5. Research alternatives 

6. Analyze alternatives by trade-off matrix 

7. Identify a potential solution 



8. Research in detail the potential solution 
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9. Design a potential solution 

10. Construct a prototype 

11. Evaluate prototype 

12. Reiterate if necessary 

13. Simplify if possible 

Although the Dartmouth engineering design process is significantly more verbose than 
Dewey’s method of problem solving there are distinct similarities. Steps one to three in 
the Dartmouth process map quite well onto steps one and two of Dewey’s, steps four 
through eight onto step three of Dewey’s, steps nine and ten to step four of Dewey’s 
and steps eleven and twelve onto step five of Dewey’s. The thirteenth step of the 
Dartmouth model is not specifically addressed in Dewey’s but this step is calling for the 
evaluation and reiteration of the process, something which is frequently discussed in 
relation to problem solving methods but which does not often receive its own step. 

Of most interest to the purposes of this study is that both the problem solving 
method proposed by Dewey and the engineering design method used at Dartmouth 
specifically include steps where the problem is defined and clarified. The Dartmouth 
model goes so far as to suggest that the problem be restated, language which is not all 
that different than the reformulation of the problem suggested by Duncker (1945). This 
clarification, restatement or reformulation of the problem can be understood to mean the 
finding of the true problem as discussed in the first chapter. Thus, both the problem 
solving and engineering design processes will require problem finding or question 



posing abilities. 




20 

Collegiate engineering education researchers are also coming to understand the 
importance of problem finding in engineering. Atman, Yasuhara, Adams, Barker, Turns 
& Rhone (2008) point out the skills required of modem engineers: 

“Today’s engineers must have mank skills to succeed in the increasingly 
complex world of engineering work. These skills include, among others, and 
ability to define problems as well as to solve them, a tolerance for ambiguity, 
design judgment, an understanding of uncertainty and an appreciation of the 
impact of designed solutions on the people and environment they interact with. 
Because engineering is situated in real contexts, an ability to consider broad 
impacts (encompassing technical, social, economic, political, cultural and 
environmental considerations) is a particularly important aspect of being a 
successful engineer.” (p. 234) 

If these skills, echoed in Hill & Smith (1998), are to be required of successful engineers 
they will need to be taught and assessed by engineering educators. Thus, a need arises 
to find ways to assess the success of teaching engineering students to think broadly. It is 
for this reason that Atman et al. (2008) turn to research on problem finding where 
designers explore the issues surrounding a design challenge and determine what the 
boundaries are for the problem they will ultimately solve. 

Atman et al. (2008) used verbal protocol analysis to examine the factors college 
freshman and senior engineering students took into account when designing a flood 
retaining wall system for the Mississippi River. The so called ‘Midwest Floods’ 
problem was designed to be both something encountered in the real-world by engineers 
and a problem which is ill-structured. The verbal protocol of participants was coded 
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based on the physical location of factors they would consider (wall, water, bank, and 
surroundings) as well as the frame of reference (technical, logistical, natural, and social) 
by two coders. 

As a result of their study Atman et al. (2008) discovered that both freshmen and 
senior engineering students considered broad, contextual issues in the formulation of 
engineering problems. Further analysis suggested that there are differences between 
freshmen and seniors in problem scoping, both in terms of the quantity of factors 
considered and the breadth or variety of factors considered. Based on this and other 
analyses Atman et al. (2008) suggest that as engineers grow from novices to experts in 
problem scoping they begin to continually revisit problem scoping throughout the 
design process rather than concentrating most scoping behavior at the beginning of the 
process. It is further suggested that the Midwest floods problem could be useful in 
evaluating the success of design education efforts. 

In 1984 researcher Jonathan Smilansky noted both the critical importance of 
problem formulation and question posing while also noting that little research had been 
done on the relationship between problem solving and problem finding and question 
posing. In an attempt to fill this void and connect the ideas of creativity and intelligence 
Smilansky set about designing a multi-part empirical study to assess what, if any, 
relationship existed between problem solving ability and problem finding or question 
posing ability. 

One major hurdle for Smilansky’s (1984) study was that problem finding studies 
to that point had primarily utilized observation of a relatively small sample and did not 
have a scoring system conducive to large samples. This required Smilansky to develop 
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both a paper instrument which could be administered in quantity and a scoring system 
to link problem solving ability with problem finding ability. Because the investigation 
was to look at this link Smilansky wanted to remove the influence of both content 
knowledge and prior experience. To achieve this end he settled on using a variation of 
the Raven Progressive Matrices test where he first asked students to solve a series of 
matrix problems and then to invent their own more difficult matrix problem. 

Smilansky (1984) concluded that although only a very low correlation existed 
between problem solving scores and the ability to create new problems a relationship 
did exist. By noting that participants wither performed well or poorly on both tasks, or 
their problem solving ability exceeded their capacity for problem invention he 
determined that the creation of new problems is more difficult than solving existing 
ones. Smilansky further suggested that the ability to solve problems was a necessary but 
not sufficient condition for inventing new problems. 

These results indicated to Smilansky (1984) that two different intellectual 
processes were taking place. He further suggested that creativity is centered on the 
ability to pose high level problems while intelligence is reserved for the ability to solve 
problems identified by others. In this way intelligence would be seen as the ability of an 
individual to apply knowledge and experience as a tool while creativity is seen as the 
more desirable and higher order ability to formulate the problem (Smilansky, 1984). 
Problem finding in schools 

Problem finding is not solely the domain of cognitive psychologists and others 
studying the problem solving process. Within the subjects of math and science problem 
finding and question posing have been identified as legitimate methods for teaching and 




learning. In the 1960s mathematics researchers Stephen Brown and Marion Walter 
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taught a series of courses on problem posing at the Harvard Graduate School of 
Education (Brown & Walter, 1990). These courses were initially only designed as a 
small extension of the work on problem solving in math by George Polya but turned out 
to be the beginning of extensive interest in problem posing. In the subsequent decades 
Brown and Walter taught a number of seminars on problem posing at colleges around 
the country. In their 1983 and 1990 books The Art of Problem Posing Brown and 
Walter suggest that when teachers step away from the traditional format where they and 
the text are the authority and ask questions for students to answer and move towards 
encouraging students to ask questions there are profound effects on student learning. 
While they note that it has been good pedagogy to encourage students to ask questions 
for some time the types of questions that students typically ask are of the procedural 
nature and not deep or particularly insightful (Brown & Walter, 1990). The goal of their 
work is to encourage teachers to incorporate problem posing by students into the 
introduction of new material in a way which slows the students to build relationships 
between new and old concepts and cement understanding. 

Brown and Walter are not alone in encouraging mathematics teachers to utilize 
problem posing in the classroom. The National Council of Teachers of Mathematics 
(NCTM) has also suggested problem posing for use in mathematics education (National 
Council of Teachers of Mathematics, 1989; National Council of Teachers of 
Mathematics, 1991). Many additional mathematics education researchers have noted the 
central importance of problem posing in mathematics education (English, 1998; Silver 
& Cai, 1996; Silver, Mamona-Downs, Leung, & Kenney, 1996). At the same time 
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English (1998) noted that despite this importance there has been limited research on the 



ability of children to create their own problems and on the success of curriculum 
targeted at improving their problem posing skills. Silver & Cai (1996) also suggest that 
much of the interest in mathematical problem posing has been at the elementary level 
and that practitioner interest has outpaced the research on problem posing specifically 
citing the assessment of problem posing as an area of need. Another potential concern 
for mathematics educators is that much of the research that has been conducted on 
problem posing refers to problem posing as the generation and reformulation of 
problems which allows for the solving of them as was suggested in the first chapter. In 
contrast much of the interest from mathematics educators is in the actual process of 
problem posing itself and the creation of problems based on experience and curiosity 
(see Brown & Walter, 1990), an area in which there is much less hard research (Silver 
et ah, 1996). 

Mathematics is not the only school subject with interest in problem posing. 
Despite the perceptions of students that science is about the study of scientific laws and 
facts there is a strong interest in the thinking and the ability to solve conceptual 
problems requiring intellectual skills (Don & Herscovitz, 1999). Along these lines there 
is increased interest in alternative evaluation methods which would allow for the 
measurement of these skills in students. One method suggested by Dori & Herscovitz 
(1999) was the measurement of problem posing as an alternative method for evaluation 
in science. Shodell (1995) has also suggested that a primary function of science 
education should be the development of problem posing capability in students. 
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Echoing many other authors, Hoover & Feldhusen (1990) and Hoover (1994) 
suggest that the ability to formulate effective problems is a critical skill with an impact 
on subsequent student success. They argue specifically that while much of science 
education is focused on the testing of hypotheses a great area of need exists in teaching 
students to develop appropriate hypotheses, a process they suggest is the manifestation 
of problem finding in science. In an effort to explore this line of inquiry two similar 
studies were undertaken by the researchers. 

Both Hoover & Feldhusen (1990) and Hoover (1994) utilized similar methods to 
explore hypothesis generation as problem finding among gifted ninth grade (Hoover & 
Feldhusen, 1990) and fifth grade (Hoover, 1994) students. In both cases relationships 
between cognitive variables such as scores on the Cognitive Abilities Test (CAT) and 
Differential Aptitude Test (DAT) and non-cognitive variables such as scores on the 
Intolerance of Ambiguity test, the Attitude Towards Problem Solving inventory, the 
Science Attitude Questionnaire and gender were examined. The 1994 study also added 
the dimension of verbal creativity though an administration of the verbal form of the 
Torrence Test of Creative Thinking. Both studies used a written form of Frederickson’s 
Formulating Hypotheses Test, scored by a panel of judges, to assess the ability of 
participants to formulate hypotheses for several realistic scientific situations. 

In both the 1990 and 1994 study the researchers found no practically significant 
relationship between the measured variables and performance on the formulating 
hypotheses test. They also found no relationship between gender and hypothesis 
formulation ability. It was noted in the 1994 study of fifth grade students that the ninth 
grade students studied in 1990 did quite a bit better and was further hypothesized that 
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the quality and quantity of scientific problems students are able to formulate increases 
with age. 

The results of the study by Don & Herscovitz (1999) indicated that problem 
posing can both be used as an alternative evaluation method for case-based learning in 
the science classroom and the inclusion of a problem posing component to these cases 
can measurably improve student performance on measures of problem posing. Though 
not yet as extensive as the interest in mathematics there appears to be continued interest 
in the study of problem posing by science educators. 

Although the use of problem posing itself as a teaching and learning tool by 
mathematics educators is interesting and does have some potential to add to the 
literature on problem posing it is not as relevant to engineering design as the use in 
science. Much like technology educators, science educators have seen the development 
of critical thinking and problem solving skills an essential part of their curriculum. It is 
in this effort to encourage scientific thinking and conceptual understanding that science 
educators have become interested in problem finding. The research by Dori & 
Herscovitz on problem posing as an alternative evaluation method might be of special 
interest to the engineering and technology education field. Similarly, the research by 
Hoover & Feldhusen (1990) and Hoover (1994) on hypothesis development, a task 
central to science and which they suggest is missing in science education, suggests that 
engineering problem finding may be a critical but missing link in engineering education 
and warrants closer examination. 

Educational researchers Lee & Cho (2007) also looked at problem finding in 
science. Their interest in problem solving comes from the view that problem 
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identification is the first step in good problem solving and leads to more effective, 
innovative, and creative solutions. It was the goal of their study to understand what 
variables might affect scientific problem finding in an effort to determine areas which 
could be targeted to improve student scientific problem finding ability. 

To go about this exploration of scientific problem finding Lee & Cho (2007) 
measured several variables though a series of tests which they believed may have an 
effect on problem finding. The variables measured were intelligence, scientific 
knowledge, science process skills, divergent thinking, intrinsic and extrinsic motivation, 
personality traits and creative home environment. In addition, Lee & Cho (2007) felt 
that some of the discrepancies in prior problem finding research may be explained by 
how structured the situation presented to students was. In an effort to test this they 
administered two separate problem finding tests which they developed. One group of 
students received a moderately structured scenario while another received an ill- 
structured scenario. Student responses were scored by a trained panel of judges on 
measures of appropriateness, originality and elaborateness. 

The results of the Lee & Cho (2007) study indicated that students were more 
likely to find appropriate, original and elaborate problems in the ill-structured scenario 
than in the moderately structured scenario. In addition, the effect of the measured 
variables changed based on how structured the presented scenario was. One of the 
interesting findings was that in the ill-structured situation scientific knowledge was a 
predictor of problem finding performance (Lee & Cho, 2007). Based on this finding Lee 
& Cho encouraged curriculum developers to consider scientific knowledge in 



curriculum designed to boost scientific problem finding performance. Conversely, ill- 
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structured problem finding performance may be indicative of knowledge and useful for 
knowledge assessment. 

One of the major issues teachers have with problem based learning and the type 
of team oriented design problems that are encouraged in engineering curriculum is the 
difficulty in assessing student performance. Design is so often a subjective assessment, 
something that ean be difficult to justify to parents and students even when rubrics are 
used. Furthermore, the group process which is frequently used in these scenarios can 
leave a teacher with little information about the thinking which went into a design and 
which might be of use in determining a grade. The use of problem posing as an 
alternative evaluation method has the potential to give teachers some additional 
information about student thinking and the creative process which led them to a specific 
design, something which would be of certain interest for instructors who are frequently 
asking students how they arrived at a particular design. 

Creativity 

Until Guilford published his seminal work on creativity in 1950 psychologists 
and cognitive scientists had largely ignored it as a vein of research (Getzels & 
Csikszentmihalyi, 1976). Since that time there has been an explosion in creativity 
research which has frequently centered on problem solving (Weisberg, 1988). Despite 
this focus much of the research was conducted using the extremely well structured and 
defined puzzle type problems which are not conducive to an examination of creative 
problem solving (Jonassen, 2000). Furthermore, it is often the case that researchers 
looking at problem solving will largely neglect creativity in their study. For example the 
seminal study of human problem solving by Newell & Simon does not even include 
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creativity in the index. Still, the interest in linking problem solving and creativity 
remains strong. Noted researcher on creativity and developer of the Torrance Tests of 
Creative Thinking E. Paul Torrance (1966) states that: 

“Creativity is a process of becoming sensitive to problems, deficiencies, gaps in 
knowledge, missing elements, disharmonies, and so on; identifying the difficult; 
searching for solutions, making guesses or formulating hypotheses about 
deficiencies, testing and re-testing these hypotheses and possibly modifying and 
retesting them, and finally communicating the results.” 

This view of the creative process is not all that different from the problem solving and 
engineering design processes we have already discussed. In fact some examples of the 
creative process seem to be derived from the engineering design and problem solving 
processes. Take for example the creative process identified at General Electric 
(McPherson, 1964, p. 133): 

1 . Define the problem 

a. Establish the problem 

b. Investigate approaches 

2. Search for methods 

3. Evaluate all methods 

4. Generalize the results 

5. Select method 

6. Make the preliminary design 

7. Perform test and evaluation 



8. Generalize the results 
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9. Find the best solution 

Although this has been identified by McPherson as a creative process it looks very 
much like an engineering design process which makes sense given the tie to General 
Electric, a firm more known for engineering than for art. Aaron Blicblau and Joseph 
Steiner (1998), in an article about creativity and engineering, suggest that it is the 
process of solving problems itself which instills the need for creativity in engineering. 
Indeed, there is much literature which links creativity with problem solving. For 
example, Weisberg (1988) explains the solutions typically given to Duncker’s (1945) 
candle problem as creative responses to a novel situation. 

The conclusions of Weisberg tend to agree with the Perkins’ (1990) discussion 
of what constitutes creativity. According to Perkins before we can understand creative 
thinking we must understand what people identify as creative thinking, usually related 
to an outcome. Also, it is not just traditional creative activities such as painting, writing, 
music and theater which can be creative outcomes. Perkins believes that creative 
outcomes can include “poems, paintings, scientific theories, business plans, jokes, 
flower arrangements, cakes, games, and conversations” (1990). By extension it would 
be possible to include the solutions to engineering design problems among these 
creative outcomes. As for what makes the outcomes the result of creative thinking, 
Perkins believes that it is ideas which are both novel and appropriate to the situation 
which are identified as creative ideas (1990). 

There has been some debate within the psychological community about whether 
or what type of relationship exists between creativity and intelligence. In his seminal 



article Creativity Guilford (1950) notes that although laypeople may believe creativity 
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is only within the grasp of gifted individuals he believes that it exists to some degree in 



everybody as we all partake in creative thinking from time to time. As evidence of this 
belief he notes that the term genius, originally used to describe the exceptionally 
creative, has been co-opted and is now used to describe those with exceptionally high 
IQs (Guilford, 1950). Furthermore, it is noted that Binet’s IQ test has been validated for 
use primarily as a predictor of academic performance in reading and mathematics and 
as such has limited applicability in the creative domain (Guilford, 1950). Despite 
Guilford’s insight into creativity and his belief that we must look beyond general 
intelligence when examining the domain of creativity he still believed that it was 
individuals with exceptional levels of fluency, flexibility and other extraordinary 
thought processes who did creative things (Guilford, 1950; Weisberg, 1988). This has 
been largely discounted by more recent research although the idea that there is 
something abnormally special about creative people continues to pervade society 
(Perkins, 1990; Weisberg, 1988). 

One of the challenges in studying creativity is that of predicting and measuring 
creative performance, especially on a large scale. Much of the older research on 
predicting creativity used standard measures of divergent thinking such as the Uses, 
Instances, and Similarities tests by Wallach and Kogan (Okuda, Ruco & Berger, 1991). 
In an effort to improve upon these measures Okuda et al. (1991) and Chand & Runco 
(1993) began to examine the use of real world problem finding as a predictor of creative 
performance. These two studies utilized written problem finding and solving exercises, 
standard measures of divergent thinking and the Creative Activities Checklist. The key 
observation of both Okuda et al. (1991) and Chand & Runco (1993) was that real world 
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problems and problem finding tasks can be significantly more predictive of 
extracurricular creative activities than traditional divergent thinking tests. 

Although variables such as personality, values, intrinsic motivation and 
discovery orientation have been suggested as predictors of creativity these, by 
themselves, leave much variance to be accounted for (Csikszentmihalyi, 1994). It is in 
this frame of mind that Csikszentmihalyi came to believe that creativity is more than 
just an individual trait and that it is based in a social and cultural context. According to 
Csikszentmihalyi the difficulty in studying creativity comes from the difficulty of 
defining it. In other words, all measures of creativity come down to a judgment call at 
some point. There is an individual or panel of individuals who must determine if 
something is creative or not. Even under the best circumstances these evaluators are 
bound by the current values and norms as they determine creativity. It is even true that 
in some cases individuals who were not considered particularly creative by themselves 
of their contemporaries are later judged to be creative based on their contribution to 
overall development. Thus, Csikszentmihalyi concluded that “creativity is not an 
attribute of individuals but of social systems making judgments about individuals” 
(1994). Although this does not reduce the importance of creativity it does mean we need 
to consider more than the individual when we examine creativity, we must think about 
the environment. 

Creativity in engineering design 

While it is the case that many engineers are initially drawn to the profession by 
an interest in gaining the ability to solve real-world problems, problems which would 



require creative solutions, these are not the types of problems which they frequently 
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encounter in the engineering schools (Blicblau & Steiner, 1998). In contrast, there is a 
concern among some engineering educators that the engineering science based 
curriculum which pervades most schools of engineering has the opposite effect, that of 
stifling creativity (Blicblau & Steiner, 1998). Noting that engineers must frequently 
work within the confines of legal requirements, cost requirements and material 
properties Blicblau & Steiner (1998) believe there remains a need to foster creativity in 
these students. One way they propose doing this is though the integration of capstone 
projects. It is their belief that these projects simulate realistic design scenarios and 
improve students’ understanding, motivation and creativity (Blicblau & Steiner, 1998). 

There is also a concern among the engineering profession that problem solving 
has become too analytical and procedural (Lumsdaine & Lumsdaine, 1994). The 
concern is that students in engineering schools are learning “engineering by formula” 
and are being taught with neat problems which will never be seen in the real world and 
which do not transfer well to the complex nature of real problems. This “plug-and- 
chug” method of engineering is not only what industries hiring engineers want to avoid, 
it is also a less efficient method of arriving at a solution (Lumsdaine & Lumsdaine, 
1994). What Lumsdaine & Lumsdaine propose is that students be familiarized with 
working in real-world contextualized situations throughout their engineering school 
careers and that creativity in design is encouraged rather than discouraged. 

This is not to say that engineering schools are all discouraging creativity or the 
creative design process. Quite the contrary, some engineering schools have made 
tremendous efforts to include creativity within their curriculum. Even going back to 
1960s there were engineering educators concerned about incorporating creativity within 
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their curriculum. In their discussion about the importance of creativity in engineering 
Mathews & Bailey (1965) discuss their course in creative problem solving for 
engineers. They note with concern the “one problem-one solution” belief among many 
engineering graduates of the day as well as the fact that virtually all leaps forward in 
engineering have been the result of someone being willing to depart from conventional 
thinking and take a risk with an original idea (Mathews & Bailey, 1965). Like 
Lumsdaine & Lumsdaine (1994) they believe that the industries which employ 
engineers are more interested in those engineers with creative abilities than those with 
only technical skill (Mathews & Bailey, 1965). They further go on to cite the existence 
and success of a formal training program in existence since 1937 at General Electric to 
encourage creativity within graduate engineers as evidence that creativity can in fact be 
enhanced by instruction (Mathews & Bailey, 1965). 

Although the course described by Mathews & Bailey stops short of encouraging 
extensive problem finding by students it does contain the seeds of such an idea. For 
example, students are asked to “Submit an idea for measuring when highway signs need 
cleaning” (Mathews & Bailey, 1965). While this pre-supposes that the problem is the 
highway department needs a system for keeping track of when signs need to be cleaned 
there are other possibilities. The creative student in examining this dilemma might 
instead ask how the signs get dirty in the first place or if there isn’t a way to reduce or 
eliminate the need to clean signs. This type of problem finding, though not explicitly 
included in the course, would seem to be encouraged as a creative solution to the 
problem. There is one project in the course where the students are expected to find and 



solve their own engineering problem. Towards the end of the course students are asked 
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to form groups and then select and solve an electrical engineering problem which has 
not yet been solved or to make a major improvement in an existing solution (Mathews 
& Bailey, 1965). 

Pappas (2002) too believes that creativity is at the heart of engineering but has 
been largely neglected by schools. Citing the increasingly complex issues which 
engineers are asked to address as well as the changing nature of technology and a 
complex society he believes that engineers can benefit from the type of creative 
thinking which has can bridge between engineering concepts and real-world problems. 
Despite the calls for innovative engineers by industry Pappas believes that engineering 
schools have been slow to respond. It is suggested that classroom activities such as 
reflection, writing, visualization, brainstorming and non- argumentative conversation 
support the development of creative problem solving skills and creative thinking 
(Pappas, 2002). 

In their study of creativity in design Dorst and Cross (2001) note the importance 
of defining and framing the problem in the development of creative design solutions. 
They further indicated that “The designer decides what to do (and when) on the basis of 
a personally perceived and constructed design task. . .The creativity of the design is thus 
influenced by all these factors.” (Dorst & Cross, 2001). In other words, the designers 
individually found and reformulated the dilemma they were given and this was a critical 
step in the design of creative solutions which evolved simultaneously with the 
designers’ understanding of the problem. 

Engineering educator and researcher Richard Felder (1987) believes that the 
most troubling problems facing society today “how to provide all our citizens with 
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adequate and affordable food, housing and medical care, efficient and economical 
public transportation, clean and safe energy” (p. 222) will require creative problem 
solving on the part of engineers. Furthermore, he suggests that those responsible for 
educating future engineers, the engineering faculty, should be responsible for ensuring a 
supply of creative engineers or at the very least not discouraging engineers from being 
creative. Yet, he believes that for all the talk of problem solving and critical thinking 
there has been little movement in engineering schools away from the traditional 
methods of doing things where students are given lectures, homework and quizzes 
which revolve around well-defined problems (Felder, 1987). Although this method for 
presenting information is efficient on the part of the instructor it does not encourage 
creativity on the part of students and leads them down the path of relying on facts and 
laws rather than the development of new and creative ideas with the potential for lasting 
impact. Along with other activities aimed at encouraging creativity within his students 
one of the questions which Felder (1987) often gives to his students is to come up with 
a problem where specific engineering concepts might prove useful. This type of 
backwards question in which students are asked to find a problem is indicative of the 
creative thinking Felder wishes to encourage. 

In much the same way that creativity has formed a backdrop for the study of 
engineering and design in the professional fields so too must creativity inform the field 
of engineering and technology education. Students seem to be both motivated and 
interested in the ability of engineering and design to solve real-world problems in 
creative ways. Curiously, one of the issues listed by Matthews & Bailey (1965) in their 



creative problem solving course was that students became too interested and involved in 
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their course to the extent that excessive time was being spent and other coursework 
neglected. Although there has been a focus on engineering science in schools of 
engineering there is some evidence of backlash against this and technology education 
must be careful about embracing the fact-based and exact nature of science and 
engineering when the ill- structured and complex problems which we expect and hope 
that engineers are able to solve are frequently aided by the inclusion of creativity rather 
than mere technical skill. We must also see the motivational nature of creativity in the 
curriculum; one must only look as far as the children’s literature section of a library and 
the popularity of books and projects in the style of Rube Goldberg to see that the 
interests of children are captured by the creative solutions to everyday problems. 
Creativity in art 

Perhaps the most quintessential discussions of creativity revolve around art. 
Artists and musicians have been identified as “creative people” for centuries and have 
become forever tied to the literature on creativity and problem finding because of their 
selection as subjects for a landmark longitudinal study by Jacob Getzels and Mihaly 
Csikszentmihalyi (1976). Their study began as an attempt to determine what indicators 
of future success, which in the art world is often based on perceived creativity of the 
artist, might be indentified in art schools students but what they ended up with surprised 
even them. 

Getzels & Csikszentmihalyi noted that in order to truly examine the creative 
nature of artists they would need to look beyond the finished products and observe the 
creation of art (1976). Beginning with students at the Art Institute of Chicago, students 



who had made a commitment to becoming artists, they began to explore the process of 
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creativity. Although there was some existing literature on the creativity of artists it was 
at best vague and tended to describe creativity as a subconscious impulse which is 
somehow translated onto the page. Investigators of creativity had usually started by 
observing the response to a presented problem, a tactic that Getzels & Csikszentmihalyi 
decided would not work. Instead, they focused on the development of the problem 
which is often where creative thinking begins. It is in this stage of problem development 
where creativity takes hold and a problem is found which leads to a creative solution 
(Getzels & Csikszentmihalyi, 1976). With this in mind Getzels & Csikszentmihalyi set 
about studying creativity based on the problem finding of artists. 

What they discovered was more than knowledge, technical skill or 
craftsmanship what set apart successful artists from those less successful was their 
ability to find problems (Getzels & Csikszentmihalyi, 1976). Even more so than as a 
longitudinal predictor the art students who showed evidence of a problem finding 
process produced more creative works than their counterparts who viewed art creation 
as a problem solving process (Getzels & Csikszentmihalyi, 1976). 

The results of the Getzels & Csikszentmihalyi study have profound implications 
for the inclusion of engineering design in the technology education curriculum. Lewis 
(2005b) has suggested that creativity is the underpinning of much of what we do in 
technology education, particularly the design and problem solving aspects encouraged 
by the Standards for Technological Literacy (International Technology Education 
Association, 2000). Lewis (2005b) argues that design gives us reason to step outside of 
traditional well- structured problem solving approaches and into creative solutions. If we 
are indeed seeking to encourage students to find novel and appropriate solutions to the 
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dilemmas with which they are presented problem finding will need to be a critical 
aspect of the curriculum although thus far it has been largely neglected (Lewis et ah, 
1998; Lewis, 2005b). 

Summary 

The increasing importance of engineering and engineering design within the 
technology education field presents new challenges for the technology educator, 
particularly as it related to the development and assessment of new curriculum. 
Successful engineers are able to deal with ambiguity, uncertainty and understand the 
broad implications of their designs and successful engineering education curriculum 
addresses these areas. Engineering design has been presented as a specific case of 
problem solving which begins with the problem finding or scoping process and which 
drives the selection and quality of the ultimate solution. 

Another mark of quality engineers is the ability to find creative solutions. Creative 
solutions, as a goal of the design process, are encouraged through the exploration of 
creative problems. The idea of problem finding is not unique to engineering education. 
Education researchers from the math and science fields have been exploring the benefits 
and relationships of problem finding at the K-12 level for a number of years. 

Problem finding researchers have utilized many different methods of assessing 
problem finding ability. The most common assessments of problem finding ability have 
relied on paper and pencil instruments which are scored by a panel of trained judges on 
one or more measures of creativity including flexibility, fluency, originality and 



elaborateness. 




Chapter III 
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Method 

Keeping in mind that solving complex and ill- structured problems is a major 
tenet of engineering and technology education curricula and that this requires the 
problem solvers have the ability to define and question the problem itself through a 
process of problem finding it is the goal of this study to explore the influence of 
engineering and technology education courses on the problem finding ability of 
engineering and technology education students. It is this goal from which the four 
guiding research questions were derived: 

1 . How do the problems identified by students at three Minnesota high schools 
compare on known measures of creativity? 

2. Does gender make a difference in the types or creativity of problems posed by 
students from three Minnesota high schools? 

3. Does coursework or curriculum make a difference in the types or creativity of 
problems posed by students from three Minnesota high schools? 

4. What do the problems posed by students at three Minnesota high schools reveal 
about their understanding of the nature of engineering? 

This chapter will examine the methodology and procedures used in this study 
through the lens of related areas of study. The chapter consists of two primary sections. 
The first section will explore methodologies and procedures used in the past for the 
study of creativity, design, and problem posing; the three areas of research most closely 
associated with this study. The second section will provide details about the procedures 
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and method used in this study including information about the variables measured, the 
setting and the instrument used. 

Methods for studying creativity 

Interest in the measurement of creativity is something which developed 
primarily in the second half of the twentieth century. Prior to this creativity had been 
considered the domain of the genius and a mysterious process which could not be fully 
understood. If creativity was an inherited trait or one received by chance there would be 
little purpose in measuring it beyond the worth of the products created. Once creativity 
began to be viewed as a valuable social trait to which all might aspire and interest arose 
in the teaching and nurturing of creativity there was an explosion of interest in the 
measurement of creativity so that these efforts to teach and nurture creativity might be 
evaluated (Mooney & Razik, 1967). 

Before one can begin to measure creativity there must be a definition which 
clarifies what about creativity is to be measured. In Guilford’s (1950) seminal article on 
creativity he suggests that creative behavior consists of activities such as inventing, 
designing, contriving, composing, and planning. Based on his work other researchers 
such as Hitt & Stock (1967) indicate that originality is the primary concern of creativity 
an idea expanded on by Goldman (1967) who describes creativity as original, inventive 
and novel ideas. 

If original, inventive and novel ideas are the outcomes of creativity we are still 
left at somewhat of a loss on how to measure these ideas as they are, at least to some 
extent, subjective measures which vary not only by evaluator but by time and place. For 
example, using the Earth’s magnetic field to tell direction was no longer novel in Asian 
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cultures by the time it was ‘discovered’ by some creative Europeans. Although it can be 
difficult to reconcile these differences in time and place when measuring creativity there 
has been much work done addressing the concerns of subjectivity. 

Much of the early work in creativity involved the use of traditional measures of 
achievement such as school grades and traditional intelligence testing (Taylor & 
Holland, 1967). From a procedural standpoint these would make ideal measures of 
creativity as there had already been significant research on measures of educational 
assessment. Unfortunately, both school grades and intelligence proved to be poor 
predictors of creativity as they do not often center on or reward creative activities 
(Taylor & Holland, 1967). Instead, there would need to be new measures of creativity 
developed. 

After much analysis of intelligence tests and their relation to creativity 
researchers such as Guilford and Thurstone began to see creativity as the combination 
of multiple factors which could be measured separately (Taylor & Holland, 1967). 
Guilford (1959) identified some of these factors as originality, adaptive flexibility, 
spontaneous flexibility, ideational fluency, expressional fluency, associational fluency, 
word fluency, sensitivity to problems, visualization, judgment and redefinition. 
Guilford’s factors, to one degree or another, seem to have been generally adopted as 
measures of creativity by other creativity researchers (Burkhart, 1967; Goldman, 1967; 
Taylor & Holland, 1967). 

With the measures of creativity defined researchers could develop specific tests 
for creativity. Taylor & Holland (1967) describe the use of word association, describing 
uses for things, identification of hidden shapes, fables, makeup problems, inventive 
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manipulation, identification of alternative solutions to frustrating situations in well 
known children’s stories, ask-and-guess tests and match problems as specific tests 
which have been used to measure creativity. These tests are not without problems 
though. Goldman (1967) notes that tests of creativity which encourage divergent 
thinking are often difficult to score because there is not a constrained set of factually 
correct answers. Guilford (1967) underscores this difficulty and notes that much of the 
measurement of creativity relies on the subjective judgments of observers. One method 
of improving reliability suggested by Goldman (1967) and Guilford (1967) is the use of 
a panel of well-instructed raters on measures of creativity. This solution was 
implemented as jury panels by Getzels & Csikszentmihslyi (1976) in their seminal 
study on creativity in art. 

Methods for studying engineering design 

Although many researchers have studied the engineering design process, much 
of their research has been confined to structured interviewing, observation and verbal 
protocol analysis (Atman, Chimka, Bursic, & Nachtmann, 1999; Cross, 1999). While 
these types of studies can provide a great depth of knowledge about the design process 
they tend to focus on understanding the process through a very small group of designers 
and not on surveying the abilities of a large group. The goal of this type of research 
seems more to be the identification and exploration of a design process or model than 
determining who is good at design or why they are. Furthermore, although there is 
recognition among design scholars that engineering design is a type of problem solving 
there have been few studies which specifically examine this link or discuss the problem 



finding aspect of engineering design. 
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The most common method for studying the design process appears to be think 
aloud verbal protocol analysis (Atman et ah, 1999; Dorst & Cross, 2001). This type of 
research involves participants verbalizing thoughts and actions as they work through a 
design problem. Instead of asking the participant to reflect on the design process and 
thus distract them and introduce their own thoughts the think aloud process is designed 
to provide as little interruption as possible and allow conclusions to be drawn about the 
cognitive processes being employed by the participant as they go about their task. The 
recorded protocols are transcribed then segmented and coded by the researcher in an 
effort to analyze the design process or methodology. 

A typical verbal protocol analysis study such as Atman et al. (1999) includes 
coding for the design step, current activity of the participant, type of information being 
processed and object being considered. These coded protocols are then analyzed 
depending on the specific nature of the research questions. For example, in the Atman et 
al. (1999) study they were primarily concerned with differences in the design stages 
used by freshmen and senior engineering students as well as the amount of time 
students spent in each of these design stages. 

Methods for studying problem finding 

Research on problem finding has not been solely the domain of either 
quantitative or qualitative research though there is some preference for quantitative 
methods. The preference is understandable when it is considered that research on 
problem finding grew out of research on problem solving undertaken by educational 
psychologists. Even those studies which employ data collection methods often 
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associated with qualitative studies sueh as interviews and observation are frequently 
quantified for analysis of problem finding. 

Much of the foundational researeh in problem finding has been condueted using 
what might be ealled quantified observation. This teehnique relies on observation of the 
problem finding proeess in a way whieh is eventually quantified for further analysis. 

For example, Allender (1969) studied the problem finding ability of elementary sehool 
students using this technique. 

In the Allender study the students became the simulated mayor of a small town 
and had to address those issues common to this line of work using the doeuments and 
information provided by the researcher. The students were observed as they went about 
the process of playing mayor and the amount of time they spent looking at various 
documents was recorded by observers. These ‘inquiry times’ were then analyzed by the 
researeher who drew conelusions about the willingness of students to engage in problem 
finding behavior without external feedback. 

Another example of quantified observation ean be found in the Getzels & 
Csikszentmihslyi (1976) study on problem finding in art. Participants in this study were 
observed as they created artworks from still life and their problem finding was 
quantified at various stages of the process. At the problem formulation stage problem 
finding was evaluated based on the number of objects manipulated, uniqueness of the 
objects chosen and exploratory behavior during selection and arrangement. The 
uniqueness score was inversely related to the number of artists who ehose to work with 
particular objects so that a higher seore eorresponded with objeets less frequently used 
and exploratory behavior was seored based on a rubric where the more closely an artist 
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examined potential objects the higher their score (Getzels & Csikszentmihalyi, 1976). 
Other scores such as openness of problem structure, discovery-oriented behavior, 
changes in problem structure and content are recorded in a similar manner. Although 
Getzels & Csikszentmihslyi did conduct interviews with their participants related to 
problem finding the answers to their questions were quantified for further analysis. 

Not all research in problem finding is strictly observational or reliant on 
interviews though. In a thesis study at Griffiths University in Australia researcher Peter 
Tracy (2005) utilized verbal protocol analysis to study the problem finding ability of 
high school industrial design students in Queensland schools. Participants in this study 
were given an initial dilemma which they were asked to address by designing a device. 
As they worked through the design process they were asked to verbalize their thinking 
and this was recorded for transcription and later analysis by the researcher. 

Although many researchers investigating problem finding including Getzels & 
Csikszentmihslyi (1976), Tracy (2005) and Allender (1969) utilize some from of 
observation or manipulative task to look at problem finding there have been alternative 
methods used. The most common alternative to these methods are paper and pencil 
instruments designed to elicit problem finding behavior from subjects. Paper 
instruments have been used by Chand & Runco (1993), Hoover (1990; 1994), Okuda, 
Runco & Berger (1991), Smilansky (1984), Lee & Cho (2007) and others. While the 
earlier work by Smilansky (1984) asked participants to develop progressive matricies 
test items which use simple geometric pattern matching. The thinking behind this was 
that it would eliminate prior knowledge and language skills which could complicate the 



measure of problem finding ability. 
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Just as with the study of problem solving which initially focused on well defined 



and well understood puzzle-type problems in an effort to understand the nature of 
problem solving but from which grew a later pushback from researchers who saw this 
as unauthentic and not transferring well to real world problem solving of ill-defined 
problems so has a pushback been seen in problem finding. Not all researchers agree that 
creating problem finding tasks devoid of context will significantly advance our working 
understanding of problem finding ability. Okuda, et al. (1991) followed by Chand & 
Runco (1993) make specific mention of using real-world problems which participants 
can relate to as part of their research strategy. Along similar lines, the work of Lee & 
Cho (2007) asks students to devise a problem for study given a broad area of inquiry. In 
this case students are familiar both with the material which is locally relevant and the 
devising of a problem for study which is frequently needed by students in the school 
setting. 

Design of this study 

As has been noted in previous chapters there is a great interest in developing the 
real-world problem solving ability of students, something which is seen as imperative 
both for their individual success as well as our success as a nation. If students are to 
fully realize their potential as creative problem solvers they must have a school 
experience and curriculum which supports that goal. As we have also learned, the 
finding of the problem to be solved given an initial dilemma is a key aspect which sets 
the tone for the entire problem solving process as well as the eventual success or failure 
of the solution. Therefore, it is critical that a method exist for the comparison of 




48 



problem finding across curriculum and environments which purport to aid in student 
learning of critical problem solving. 

Variables and instrument design 

Before this study could be undertaken a suitable instrument and method for the 
evaluation of engineering problem finding ability needed to be developed. The design 
of this instrument was driven by several factors. First and foremost, the data collected 
needed to support answering the three guiding research questions. This necessitated the 
collection of certain demographic data including gender and coursework history for 
engineering, advanced science, and technology education classes taken in high school. 
It also meant devising some method of capturing and measuring the ability of students 
to find engineering problems. Secondly, the design was influenced by the need to gain 
acceptance of both the University of Minnesota IRB and individual school and district 
gatekeepers. This suggested a completely anonymous data collection method and 
instrument. A copy of the instrument is included in Appendix A. 

Because of the lack of prior studies on engineering problem finding the 
development of this section was a combination of the many methods used in the past to 
study both engineering design as well as creativity. From engineering design literature 
came the idea of presenting design challenge scenarios and asking students to identify 
problems that they might choose to address if they were an engineer faced with the 
situation. From the creativity literature came the method for converting student 
responses into quantitative data, a process which will be described in more detail later. 

The two scenarios themselves were selected for several reasons. As suggested 
by Hill & Smith (1998) authentic educational experiences are critical for students. This 
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means that the scenarios should be both realistic and familiar to students. The scenarios 



must be accessible to the participants, in this case high school students from a variety of 
backgrounds. This meant that the scenarios should be understandable and common 
enough that students would have a good idea about what issues might come up. 
Secondly, they should be interesting to the participants in order to engage them in the 
problem finding exercise as much as possible. Thirdly, they should be issues that might 
be realistically encountered by a practicing engineer. Fourth, the issues should not be so 
complex or numerous that they cannot be fully discussed by an average student in a 
single class period. 

These four criteria drove the selection of two scenarios to be included in the 
instrument. Both were selected because they fit all three of the criteria reasonably well. 
The first, an issue related to energy efficiency, was both timely and accessible because 
of rising energy costs and frequent media attention to the problems that wrought. It is 
also well known that engineers address issues of energy efficiency. Indeed, many 
engineering schools take part in the US Solar Decathlon event which specifically relies 
on energy efficiency as a measure of success. 

The second scenario selected related to the river flooding of a fictional town. 
Students in Minnesota, where this study was administered, are frequently made aware 
of flooding problems along rivers by the media and many may have had personal 
experience with some kind of flooding. In addition to it being well known that 
engineers work on hydraulics and water problems respected engineering design 
researcher Cynthia Atman has used the issue of flooding to examine the design process 



of engineering students. 
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Data from five variables were captured through the paper and pencil test 
instrument which allowed for more students to participate and facilitated the later 
scoring of responses. The two primary areas into which these variables fall are 
background demographic data and measures of problem finding ability. Gender was one 
of the demographic factors useful in describing the type of problems identified by 
participants and to account for additional variability within the statistical model. 

Schools and students 

Although no identified literature has examined the differences in problem 
finding ability based on school environment there is some anecdotal evidence that 
differences may exist. In discussing the dearth of applicants to engineering schools 
many engineering faculty lament the decline of the so-called ‘farm mechanic’ which 
made up much of their admissions in the last century. The belief is that rural students 
with farm experience often need to make do with what is available to them as they go 
about solving a variety of engineering design problems such as the flat tire problem 
discussed in the first chapter. Urban and suburban students would be more familiar with 
getting things done the ‘right way’ and often have more nearby resources to draw on 
making them less apt to devise creative solutions to problems. Students from three 
different schools were administered the problem finding test in an effort to obtain 
results from different types of student populations. 

School 1 was an outer ring suburban high school from the Minneapolis/St. Paul 
area which participates in the Project Lead the Way engineering curriculum as well as 



offering advanced science and traditional technology education classes. The students 
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taking part in this study came from one of the Project Lead the Way classes though 
some had previously taken advanced science or technology education classes. 

School 2 was a rural/small town high school in south central Minnesota which 
does participate in Project Lead the Way, but only on a limited basis. When data for this 
study was collected in Fall 2008 there were no Project Lead the Way classes currently 
underway and the students came from two sections of traditional technology education 
classes. 

School 3 was a first ring suburban high school from the Minneapolis/St. Paul 
area which does not currently participate in Project Lead the Way and offers no 
technology education classes of any kind at the high school level. The school is known 
for high academic standards and has been a leader in Advanced Placement and Enriched 
science education. Students from this school came from two sections of Advanced 
Placement Physics. 

The third demographic variable collected was the type and number of courses 
and curriculum the participant has been exposed to. Students were asked to list all of the 
engineering, advanced science courses and technology education courses they have 
taken in high school. Many of the engineering and technology education courses 
purport to teach problem solving skills which have been identified as important 
outcomes but there is little known about if students who take these courses actually do 
better than their peers when faced will an ill- structured real-world challenge. 

Specific details about the participant makeup is included in Table 1, Table 2 and 



Table 3. 




Table 1. 



52 



School Demographics for Scenario 1 







Number of 
Engineering 
Classes 


Number of 
Advanced Science 
Classes 


Number of 

Technology Education 
Classes 




N 


Mean 


Std. 

Deviation 


Mean 


Std. 

Deviation 


Mean 


Std. 

Deviation 


School 1 


21 


1.52 


0.680 


0.76 


1.044 


1.38 


1.322 


School 2 


33 


0.00 


0.000 


0.00 


0.000 


1.55 


0.833 


School 3 


37 


0.00 


0.000 


3.16 


1.482 


0.00 


0.000 


Table 2. 
















School Demographics for Scenario 2 














Number of 
Engineering 
Classes 


Number of 
Advanced Science 
Classes 


Number of 

Technology Education 
Classes 




N 


Mean 


Std. 

Deviation 


Mean 


Std. 

Deviation 


Mean 


Std. 

Deviation 


School 1 


21 


1.52 


0.680 


0.76 


1.044 


1.38 


1.322 


School 2 


33 


0.00 


0.000 


0.00 


0.000 


1.52 


0.834 


School 3 


36 


0.00 


0.000 


3.19 


1.527 


0.00 


0.000 




Table 3. 
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Gender Demographics 



Number of Number of Number of 

Engineering Advanced Technology Education 

Classes Science Classes Classes 



N 



Mean 



Std. 

Deviation 



Mean 



Std. 

Deviation 



Mean 



Std. 

Deviation 



15 .13 .352 3.13 1.807 0.20 0.561 

75 .40 .771 1.15 1.608 1.01 1.121 

15 .13 .352 3.13 1.807 0.20 0.561 

74 .41 .775 1.14 1.633 1.01 1.116 

Note. One student failed to report a gender and so is excluded from this table. 

Einally, student identified problems were collected from the two simple but real- 
world and ill-structured situations described above which call for engineering design 
and problem solving. Participants were asked to identify problems in these situations 
which, if solved, would significantly improve things. 

Reviewers 

The creative problem finding ability of participants was evaluated based on the 
same measures used by Eee & Cho (2007) and which are further supported by the 
methodologies commonly used by creativity researchers. The problems identified by 
participants were evaluated by a panel comprised of researchers and educators familiar 



Scenario 1 
Eemales 

Scenario 1 
Males 

Scenario 2 
Eemales 

Scenario 2 
Males 
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with engineering design or technology education following detailed instructions, rubrics 
and training. 

Reviewer 1 was a female graduate student from the University of Minnesota 
department of Mechanical Engineering. She has been a teaching assistant in the 
undergraduate mechanical engineering capstone design course for some time. 

Reviewer 2 was a male former high school Project Lead the Way teacher now 
teaching middle school Project Lead the Way classes. As a high school teacher he spent 
several years teaching the Project Lead the Way “Introduction to Engineering Design” 
class and is also familiar with technology education. 

Reviewer 3 was a male graduate student from the University of Minnesota 
department of Mechanical Engineering. He also has been a teaching assistant in the 
undergraduate mechanical engineering capstone design course for some time. 

Each of the two scenarios completed by each participant was scored 
independently by the three judges on four measures of creativity. Judges were instructed 
to read through a sample of student responses to gauge the range and types of responses 
they would encounter during the rating process. The actual rating process had five key 
steps for each scenario. The complete judge instructions are included in Appendix D. 

1 . After reading the entire student response for the given scenario the judge 
gave the student an overall rating on fluency with assistance from a rubric. 
Judges were instructed to rate fluency based on the number of problems 
identified by the student compared to all other students for the given 
scenario. This rating was on a five point Likert type scale. 
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2. Judges then determined the number of broad categories the problems fell 
into and rated the student on how many different categories their problems 
came from. Judges were instructed to rate, flexibility based on the number of 
problems identified by the student compared to all other students for the 
given scenario. Again, a rubric was provided to assist judges. This rating 
was on a five point Likert type scale. 

3. For each of the individual problems identified by the student the judge first 
determined if it was an appropriate problem for the given scenario. This 
rating was a Yes/No dichotomous variable. Those problems which did not 
appropriately address the scenario were removed from the study. This 
follows the advice of Lee & Cho (2007) who note that it is easy for 
evaluators to determine whether a problem is appropriate but extraordinarily 
difficult to assess and place a score on the degree of appropriateness. 

4. Assuming that the individual problem was appropriate judges were asked to 
rate the originality of the problem compared with other students. The more 
common a problem was the lower the originality score judges were to give 
it. This rating was on a five point Likert type scale. 

5. Finally, judges were asked to evaluate how thoroughly the student 
elaborated on the specific problem they identified. This rating was on a five 
point Likert type scale. 

Getzels & Csikszentmihslyi (1976) noted in their study the usefulness and 
appropriateness of evaluation panels for research on problem finding. Similarly, 



Guilford (1967) noted that most research on creativity involves some measure of 
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subjectivity and is dependent on the judgments of others but that these judgments have 
been more predictable than most other criteria. The data on interrater reliability, 
calculated as Cronbach’s alpha following the data collection period, seems to bear out 
this for the most part. Interrater reliability is presented in Table 4. 

Table 4. 



Interrater Reliability Statistics 









Cronbach’s Alpha 






Cases 


Fluency 


Flexibility 


Originality 


Elaborateness 


Scenario 1 


91 


.917 


.814 


.703 


.836 


Scenario 2 


90 


.922 


.835 


.877 


.896 



Note. N =3 reviewers for each scenario. 



With the sole exception of originality in scenario one all of these alpha statistics 
meet the .800 level frequently suggested as a standard for interrater reliability. Given 
the subjective scoring utilized by the panel and the diverse backgrounds of panel 
members these values are quite good. 

Data analysis 

Analysis of the data collected from students and, ultimately, judges was 
complicated significantly by the free response nature of the ill- structured situations. In 
an effort to capture as clear a picture as possible from the data available two distinct 
approaches were used. 

First, the panel of judges was asked to rate the problems identified by students 
on the four factors of creativity: flexibility, fluency, originality and elaborateness. The 
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student response sheets provided to judges contained none of the demographic data 
about students and the only indentifying marks were unique student identifiers which 
were assigned by the researcher. The judges were provided with both written and oral 
instructions as well as a rubric to assist them in this task. A copy of all scoring 
instructions and this rubric are included in the appendices. 

Because students did not specifically number or list all of the problems 
identified and also may have included sketches, drawings or diagrams it was up to 
judges individually to determine where to separate problems identified by students. This 
led to different judges determining a different number of problems for the same student 
response. While this was not problematic for the flexibility and fluency scores which 
are per student, per scenario it was something that needed to be dealt with for originality 
and elaborateness scores which were per problem. The solution eventually employed 
was to average the originality and elaborateness scores across all appropriate problems 
identified by the student. In this way for each scenario each student ended up with a 
single score for each of the four characteristics (fluency, flexibility, originality and 
elaborateness) from each judge. 

These scores were subsequently entered into a spreadsheet along with unique 
identifiers and quality checked. At this point judges were contacted about any missing 
data and asked to re-score those students for whom they were missing data. Most of the 
judges responded promptly and those corrections were made to the spreadsheet. One of 
the judges was unable to provide corrected data and so all students missing data from 
that judge were removed from the data set. Specifically, this meant excluding two 



students from the first scenario and four students from the second scenario because one 
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or more of the creativity characteristics were unscored. This does mean that while the 
same students responded to both scenarios the students included in the quantitative 
analysis are slightly different for the two scenarios. 

Following the quality check of the data it was copied into SPSS for interrater 
reliability analysis. At this point each student in each scenario had three scores for each 
of the four creativity characteristics, one score from each judge. The RELIABILITY 
program was used within SPSS to determine the interrater reliability statistics given in 
Table 4. Once interrater reliability had been ascertained the statistical analysis could be 
continued. 

Because the research questions revolve around differences due to the 
demographic factors of school, gender and coursework history and not around 
differences between judges the data needed to be further manipulated to arrive at a 
single score for each creativity characteristic for each student in each of the two 
scenarios. This was done by averaging the three fluency, flexibility, originality and 
elaborateness scores for each participant. In this manner each student ended up with a 
single value for each of the four creativity characteristics which, when combined with 
the demographic data, could be further analyzed. 

The first statistical analysis employed was the MANOVA procedure which 
examined differences in the four creativity characteristics based on the school and 
gender of participants. Secondly, differences because of coursework history were 
explored using the multiple regression procedure because of the continuous nature of 
that variable. Both procedures were carried out independently for the first and second 
scenario. The results of these procedures can be found in the following chapter. 
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In addition to the statistical analysis of data from the judges a more qualitative 
method was used to evaluate common categories among responses by categorical 
variables. These qualitative observations were meant to be purely exploratory in nature 
but to catch any possible differences in problems found at the different schools as well 
as by female and male participants. For the first analysis student responses were sorted 
by school. All responses for one scenario at a time were read by the researcher and 
notes taken on the broad themes which seemed to categorize the majority of responses. 
This type of fuzzy and subjective analysis was required because of the many ways and 
many levels of detail with which students identified problems. At the same time specific 
examples of problems within these broad categories which exemplified the category 
were recorded for use within the results section. This process was repeated for each of 
the three schools and for the second scenario. 

Once this was complete the student responses were re-sorted based on the 
gender of the participant. The same process was employed for determining the broad 
categories of responses within each gender. 

After a complete list of common problem categories was determined for each 
school and gender by scenario the lists were compared for similarities and differences. 
The list of categories itself was coded to identify both similar and unique categories of 
problems amongst the three schools and again between the two genders. These coded 
category lists were used during the discussion of qualitative results in the following 



chapter. 




Chapter IV 
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Results 

The data were collected from each of the three schools and turned over to the 
three judge panel as described in the preceding chapter. In addition, student responses 
were qualitatively evaluated for response categories by the researcher as described in 
the previous section. Each of the two scenarios was treated separately and is presented 
here in that way. First, the energy efficiency scenario will be examined through the lens 
of the research questions. This will be done both qualitatively through a discussion of 
categories in student response and quantitatively through MANOVA and Multiple 
Regression statistical analyses. Next, the flooding scenario will be examined in the 
same way. 

Scenario 1: Energy Efficiency 

Students from School 1 and School 3 had generally similar ideas about improving 
the energy efficiency of homes while students from School 2 took a rather different 
direction. There were some ideas which were mentioned frequently by students in all 
three schools. Specifically many students mentioned installing or upgrading insulation 
as a key method for reducing energy consumption. Beyond this; however, there were 
some distinct differences. 

While the scenario presented to students specifically mentioned that the houses 
were in the design stage, i.e. not yet built, only students in schools 1 and 3 frequently 
suggested modifying the architectural layout or design of the homes. Specifically 
students from School 1 suggested design modifications such as reducing ceiling height 
to reduce the volume of conditioned air and students from School 3 suggested designing 
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homes to better take advantage of sites such as through the use of passive solar heating 



in the winter. Students from both schools mentioned the overall size of homes as a 
contributing factor to energy inefficiency and suggested redesigning homes to be 
smaller and more efficient. Table 5 and Table 6 present example student responses from 
schools 1 and 3 for this scenario organized by categories found in the responses of the 
school. 

Table 5. 



Scenario 1 Example Student Responses from School 1 by Theme 



Theme 



Example Responses 



Alternative energy 



Insulation 



Redesign for better 
heating and cooling 
efficiency 



• “Energy could be saved by using solar power on the 
roofs. . .a stream or river could help power the homes 
as well” 

• “The cost of electricity can be high. This problem 
can be solved in a variety of ways. A good way to 
solve this is to install a wind turbine (or a few wind 
turbines) that can provide electricity for an entire 
neighborhood or city. In this way, the cost will be 
spread among many people.” 

• “New energy ideas can also be explored. If at all 
cost efficient the firm could look into solar or wind 
power (or others) to power the house.” 

• “To improve the energy efficiency of the homes you 
could make the walls thicker so you can put more 
insulation in them” 

• “Heat rises therefore the first step to energy 
efficiency is to make sure the roof is well insulated” 

• “The house blue prints must try to make a design 
that will be easy to heat, cool & insulate” 

• “A big problem with energy efficiency is heating 
and cooling costs. The best way to deal with that is 
build a house with a minimal amount of exposed 
surface area...” 

• “The house should also be south facing if possible to 
help with heating costs in the winter” 

• “The very tall ceilings are a problem when the 
homeowners are trying to heat and cool the houses. 
Because heat rises, it will build up from the top 
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Heating/V entilation/Air 
Conditioning (HVAC) 
efficiency 



Things left plugged in or 
turned on 



down and will take much more time and energy to 
keep the house heated.” 



• “As for cooling, cold air will sink, so although the 
area that the people are lining in is cool, the air will 
continue pumping in until the full room is cool. I 
would advise lower ceilings, but since the house is 
already built, placing the air vents half-way down 
the wall, instead of at the top so the cold air will not 
need to fill the whole room would be a good start.” 

• “The efficiency of the furnace -i- AC is also 
important” 

• “If the home buyer wants to be able to keep the 
whole house heated, then the heating system should 
be lower or on the floor as shown in the poor 
drawing. If the vents are low the heat will have to 
travel through the air and up to the ceiling.” 

• “Problem - appliances and electronics such as TVs 
and computers still draw power when they are 
turned off or nobody is home. Solution - to prevent 
devices such as chargers from drawing power when 
not used, create some sort of device which plugs into 
an outlet which allows other cords to be plugged 
into it. When the amount of power being used drops 
below a certain threshold, the device cuts off power 
consumption completely, preventing electricity 
going to waste. To prevent power consumption from 
something such as a light being left on when nobody 
is home, create a device that allows the user to in 
some way be reminded of what is using power in 
their home, and if possible, remotely terminate the 
thing using power.” 

• “Have multiple light switches for same thing to 
increase ease of shutting off lights for occupants so 
as not to waste electricity.” 

• “Another issue making them inefficient could be the 
fact that the heat is left on even when there are no 
people home. A simple solution to this is to have a 
programmable thermostat installed in every home 
and encourage the homeowner to use it.” 




Table 6. 
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Scenario 1 Example Student Responses from School 3 by Theme 



Theme 



Example Responses 



Alternative energy 



Insulation 



Appliance efficiency, 
mostly related to HVAC 
systems 



• “Gas is very expensive, due to the economic 
downturn and high energy use. In addition, heating a 
cooling is inefficient (especially through use of gas). 
Instead, solar panels should be utilized on the roof of 
the house. They are more environmentally friendly 
and make houses look more fashionable. Renewable 
energy could heat and provide power to the home.” 

• “Solar power panels on roofing if there is enough 
sun. Nearby wind farms = renewable energy source, 
hydroelectric power. All of these would give more 
room for increasing efficieny.” 

• “Invest in solar panels. By producing its own clean 
energy during the day the house could minimize 
dependency on unclean, expensive energy services.” 

• “Heating is a big energy cost so if we cut down on 
loss of energy to the outside we can save money, 
adding insulation would help this problem” 

• “Houses should have a better insulation structure to 
keep heat in when it’s cold and expel heat when it’s 
warm to reduce the reliance on less efficient air 
conditioning units/water heaters.” 

• “Work on creating a more insulated house, double 
front doors for the trapped air in between to insulate 
the house, and the double paned glass throughout the 
entire house for more pockets of air/insulation.” 

• “Using geothermal heat to heat and cool the house 
would save energy. Since the heat of the earth below 
the surface is relatively steady in comparison to the 
fluxuating temperature above the surface. So if we 
can pump water down through tubes, have it heat up 
we can raise it back up to hear our homes cutting 
down on energy costs. It can also be used to cool 
houses in the summer.” 

• “The methods by which homes are heated and 
cooled remain extremely inefficient, despite 
technological advancements. Individual boilers to 
heat homes requires the constant heating of water, 
even if not used. Instead, one could implement a 
communal boiler, much like power from a power 
plant.” 
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Home size 



Occupancy sensing 



• “Architectually, design houses in such a way that the 
source of energy is close to the center, so less is lost 
as it gets to its destination. Put air conditioner/heater 
in the middles of the house rather than blocked off in 
the basement. Make houses more compact so the 
energy has to travel less far.” 

• “The best way to conserve energy would be reduce 
house size (reducing leaks of heat).” 

• “Problem: houses are too big Solution: build homes 
with less air space, lower ceilings, etc.” 

• “As for actual construction, houses should be built 
smaller, this would require less materials, and less 
energy to power.” 

• “Lights could also be controlled by vocal 
commands. It would be easier for homeowners to 
turn off lights, so they would conserve more 
energy.” 

• “Lighting: They could have motion detectors for 
rooms so that they turned on only when someone 
enters and then turned off when there isn’t 
movement.” 

• “Also, having a way to turn the heat down when 
people are at work or away would save energy. If 
there is a way for the heat to turn back on at a 
predetermined time then people could come home 
and never have to deal with the heat. It could save 
energy and not be a hassle to the owner” 



On the other hand the responses from School 2 tended to be briefer and more 
focused on things that a homeowner could change after the home was built rather than 
on changing the design of the home. For example, many students from this school 
suggested alternative lighting technologies, specifically compact fluorescent and light 
emitting diode based lighting as replacements for traditional tungsten light bulbs. These 
students also frequently discussed changing out windows for more efficient ones and 
ensuring that contractors did not cut comers when building the home. Overall, the 
responses from School 2 were less thorough in the quantity and quality of responses as 
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well as focusing more on behaviors of the homeowner and builder than on the homes 
themselves. Table 7 presents example student responses from school 2 for this scenario 
organized by categories found in the responses of the school. 

Table 7. 

Scenario 1 Example Student Responses from School 2 by Theme 

Theme Example Responses 

Lighting inefficiency • “Install fluorescent light bulbs in the rooms you use 

most.” 

• “Install more efficient lighting” 

• “fluorescent lightbulbs” 

Geothermal • “geo-thermal” 

• “use floor heating - geo-thermal heating” 

• “replace all light bulbs with geo thermal ones” 

Voltage loss • “Extra cable length: Sloppy workmanship by the 

electrician wiring the house also adds to the home 
owner’s bill. Untidiness may result in additional 
cable length, adding resistance, reducing ampacity, 
and drawing more electricity” 

• “Somehow find a way to manufacture cheap 
superconductors for wiring. The less resistant the 
conductors are, the more efficient they become.” 

• “Electrical: In an old home you could replace old 
lighting recepticales and wire with new upto date 
recepticales & switches. It will meet code and have a 
better electrical efficiency rating.” 

Appliance efficiency* • “Use lower watt appliances” 

• “I would solve energy by getting efficient 
appliances” 

• “If you don’t have energy star appliances that could 
greatly affect your power consumption.” 

Insulation • “Insulation: If you are planning to buy a home you 

should make sure that there is enough insulation 
needed in the attics & ducts so you don’t have heat 
loss & energy efficiency” 

• “Better insulation to hold in more heat so you don’t 
have to use the heater as much” 



• “add more insulation” 




Window efficiency 
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• “One problem that could be fixed would be bad 
windows putting a cover over it so no heat would be 
lost. If that would be fixed then a heating bill would 
cost less.” 

• “Put in some high grade windows. If you have poor 
quality windows you won’t be able to keep heat in 
during the winter or cooler air during the summer” 

• “I would make sure all windows are replaced with 
low-e argon filled windows.” 

*This was mentioned much less than in the responses of school 3, but was a minor 

theme in responses 

Differences also existed in the responses typical of female students when 
compared to the male students across all schools. Specifically, females were much more 
likely to mention the size and design of homes as problems contributing to energy 
inefficiency than their male counterparts. The females also frequently suggested that 
things in homes such as lights, electronics and heat get left on and continue to use 
energy when not in active use. One response typical of female students who mentioned 
things which are not in active use but continue to draw power looked like this: 

Problem: Energy is lost through electronics “sleeping” and on electronics while 
people aren’t there, such as forgetting to turn off lights. 

Solution: Electronics such as TVs “sleep” while ppl don’t use them. While 
sleeping they use small amounts of electricity to be ready to turn on at a moments 
notice and don’t stop sapping energy unless unplugged. TO prevent this they 
power for those types of objects could be set on a timer, w/ no powerflow when 
ppl are sleeping or gone at work. To prevent electricity waste by lights left on 
when ppl are gone, the lights should be set to activation. Eike clap-lights that turn 
on w/ a sound these lights will stay on if they “hear” a noise every 5-10 mins, w/ 
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an option for manuel over ride if a person wants to read etc. This way if ppl leave 
the lights will turn off on their own. 

On the other hand male students were more likely to discuss the efficiency of 
lights (when they are on) and suggest replacing them with more efficient illumination 
technologies. Many of the males also mentioned windows as a great source of heat loss 
which could be improved using new technologies. Students of both genders mentioned 
problems with insulation, heating and air conditioning systems, appliances and 
inefficient energy sources. Table 8 and Table 9 present example student responses for 
this scenario organized by categories found in the responses of each gender. 

Table 8. 



Scenario 1 Example Student Responses from Females by Theme 



Theme 



Example Responses 



Insulation 



Occupancy sensors 



• “Skimping on insulation from window, to floor, to 
wall, to ceiling, to roof all cause energy inefficiency 
that ends up increasing cost to the home owner in 
the long run.” 

• “One problem a house may have with lower energy 
efficiency could be related to the heat lost due to 
poor insulation. . . Solution: I would use better 
insulation in the walls when building a house and 
get windows which better keep the heat in without 
leaking/lower amount of windows.” 

• “When the sensors did not detect anyone for 5 mins 
the timer would be part of the wiring and it would 
turn off once it hit 5 mins breaking the circuit/power 
system to the lights.” 

• “Problem - appliances and electronics such as TVs 
and computers still draw power when they are 
turned off or nobody is home. Solution - to prevent 
devices such as chargers from drawing power when 
not used, create some sort of device which plugs into 
an outlet which allows other cords to be plugged 
into it. When the amount of power being used drops 
below a certain threshold, the device cuts off power 
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Heating, Ventilation, Air 
Conditioning (HVAC) 
systems 



Home size 



Alternative energy 



consumption completely, preventing electricity 
going to waste. To prevent power consumption from 
something such as a light being left on when nobody 
is home, create a device that allows the user to in 
some way be reminded of what is using power in 
their home, and if possible, remotely terminate the 
thing using power.” 

• “Inefficient heating: Instead of having central 
heating that is transmitted via ventilation you could 
put a source of heat such as a fireplace or radiator in 
the middle of the house and build rooms around it.” 

• “Inefficient heating and energy dispersal: Since the 
heating system in a house is often not central, the 
heat goes to the rooms right around the source but 
has a hard time reaching the outer rooms. Heating 
systems should be the ‘foundation’ or ‘base’ of the 
house. The rooms should be built around that source 
to minimize unequal heating.” 

• “The heating system may be inefficient because of 
heat loss especially when the house is big. The warm 
air goes up and leaves the lower part of the house 
cold. To heat up the whole house would take many 
hours and too much gas.” 

• “The best way to conserve energy would be to 
reduce house size (reducing leaks of heat).” 

• “Efficient heating and electricity could be improved 
with solar and wind energy. I’d get solar panels on 
the roof everywhere and some wind turbines on the 
trees to be more efficient. This wouldn’t look so bad 
because it’s only the roof and you could get some 
green wind turbines so it wouldn’t look so bad in the 
trees. Also you could use the heat energy produced 
by your body to power smaller appliances like 
toasters. This could be done by making a device 
connected to your skin.” 

• “Use solar power” 




Table 9. 



Window efficiency 
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“Heating is a big energy cost so if we cut down on 
loss of energy to the outside we can save money, 
adding insulation would help this problem” 

“To improve the energy efficiency of the homes you 
could make the walls thicker so you can put more 
insulation in them” 

“Install more efficient lighting” 

“Use LEDs of CEL lights wherever possible” 

“The furnace itself needs to be looked at to make 
sure that it will function properly. Air must be 
evenly distributed throughout the house to ensure 
satisfaction. The ductwork should have as little 
bends as possible since this can cause loss of air 
movement. If bends are needed then extra 
propulsion fans may need to be installed as well.” 
“Replace low/medium efficiency fumace/water- 
heater with a high efficiency model. Use multiple 
zone heating and cooling.” 

“I would solve energy by getting efficient 
appliances.” 

“Use energy star appliances for boiler, water 
softener, etc. have them centrally located within the 
house so as not to give bias to any room in the 
building.” 

“Energy could be saved by using solar power on the 
roofs of the new homes, as well as using power 
saving lights.” 

“Solar power panels on roofing if there is enough 
sun. Nearby wind farms = renewable energy source, 
hydroelectric power. All of these would give more 
room for increasing efficiency.” 

“High ceilings & lots of windows tend to lose a lot 
of heat making heating bills expensive.” 

“The main problem would be heating. A lot of heat 
is lost through windows and door frames. One could 
install windows with 2 layers of glass in order to 
increase their insulation. They could also use better 
sealants that would allow for less heat to be lost.” 
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A multivariate analysis of variance showed no statistically significant differences 
in the four traits of creativity measured by the judges at the .05 alpha level for school, 
gender or a school/gender interaction. In addition to the MANOVA procedure the data 
was analyzed using multiple regression to explore differences in each of the four traits 
based on gender as well as the number of engineering, advanced science and traditional 
technology education classes reported by each student. The results of these multiple 
regression analyses for each of the three traits with significant coefficients at the alpha 
level of .05 are presented in Table 11, Table 12 and Table 13. The multiple regression 
analysis for originality was not significant at the .05 level. The average judge scores on 
measures of creativity by school and gender are presented in Table 10. 

Table 10. 



Scenario 1 Average Scores 





Flexibility 


Fluency 


Originality 


Elaborateness 


School 1 


2.444 


2.651 


2.679 


2.300 


School 2 


2.081 


2.131 


2.472 


1.505 


School 3 


2.558 


2.631 


2.745 


2.403 


All Males 


2.382 


2.493 


2.606 


1.983 


All Females 


2.155 


2.133 


2.697 


2.391 




Table 11. 
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Scenario 1 Fluency 





Unstandardized 


Standardized 






Coefficients 


Coefficients 


Sig. 




B 


SEB 


P 




Constant 


1.974 


.239 




.000 


Gender 


-0.733 


.331 


-.238 


.030 


Engineering 

Classes 


0.474 


.167 


.297 


.006 


Advanced 
Science Classes 


0.263 


.084 


.408 


.002 


Technology 

Education 

Classes 


0.028 


.135 


.026 


.836 


Note. R^= .188 




Table 12. 
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Scenario 1 Flexibility 





Unstandardized 


Standardized 






Coefficients 


Coefficients 


Sig. 




B 


SEB 


P 




Constant 


1.988 


.224 




.000 


Gender 


-0.617 


.311 


-.219 


.050 


Engineering 

Classes 


0.281 


.157 


.192 


.077 


Advanced 
Science Classes 


0.238 


.078 


.404 


.003 


Technology 

Education 

Classes 


0.010 


.127 


.010 


.940 


Note. R^= .146 




Table 13. 
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Scenario 1 Elaborateness 





Unstandardized 


Standardized 






Coefficients 


Coefficients 


Sig. 




B 


SEB 


P 




Constant 


1.976 


.184 




.000 


Gender 


0.120 


.255 


.050 


.640 


Engineering 

Classes 


0.375 


.128 


.302 


.004 


Advanced 
Science Classes 


0.094 


.064 


.188 


.147 


Technology 

Education 

Classes 


-0.248 


.104 


-.301 


.019 


Note. R^= .208 



Scenario 2: Flooding 

Although few of the ideas put forth by students at the various schools were 
unique to a school none of the categories were found frequently across all three schools. 
For example, students from both School 1 and School 3 suggested that one potential 
problem is having too much water flow through the river near the town center and 
proposed various water diversion schemes to route excess water around the town. 
Another problem identified by students from these two schools was buildings built too 
low which they suggested rectifying either by raising or moving the buildings. Few, if 
any, students from School 2 identified either of these as problems. One problem which 




was frequently identified by students from School 2 was a lack of dams and 
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containment of water upstream, suggesting that dams and reservoirs be built to prevent 
flooding. This was a sentiment echoed by students from School 1 but mentioned much 
less by students from School 3. On the other hand, some problem categories were 
identified by students from School 3 and School 2 but not so much by students in 
School 1 . An example of this is a lack of containment systems to protect the town such 
as floodwalls, dikes and levees. This is not to say that there were no unique categories 
of response in problems suggested by students at the different schools. Example 
responses from School 1 and School 3 are provided in Table 14 and Table 15. 

Table 14. 



Scenario 2 Example Student Responses from School 1 by Theme 



Theme 



Example Responses 



Water diversion 



Moving things to higher 
ground or putting 
buildings on stilts 



• “One idea to harness the energy is to set in place a 
trench or pipe or transport system to move mass 
amount of water to other places. A pipe set in place 
4 appr. Eeet above normal river level would 
transport water to where it was needed. . .the farm 
needs the water so it is transported there.” 

• “If it is found that the river is spring fed it may be 
beneficial to divert some of the streams to other 
bodys of water in the area. Obviously one would 
need to research and take into account the 
surrounding eco-structure when ever displacement 
of waster is involved.” 

• “Creating other smaller rivers around the flooding 
river could divert most of the water away towards a 
lake or pong. New problems could then be created 
like flooding of the lake or pond or destruction of 
the ecosystem.” 

• “Buildings that have a potential of being flooded 
could be built on stilts. These stilts will hold the 
building off the ground as water flows underneath 
them in the case of a flood.” 

• “If homes are close, they should be raised. Placed 
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Dams 



higher than the surrounding land.” 

• “Someone put buildings in the wrong spot. These 
buildings should be moved to higher ground and the 
land should not be built on.” 

• “A small dam with a reservoir upstream of the town 
would help in that the flow of water would be 
regulated and could be controlled. However, dams 
can cause problems with the natural ecosystems, so 
this problem would have to be addressed according 
to the location of the dam.” 

• “There is a spot about 10 miles upstream that would 
be a prime spot to build a dam. There was a beaver 
dam up until a few years ago here, but it has since 
been destroyed and the beavers have left, which I 
believe is the reason that there is now an issue with 
flooding that was no there before.” 

• “Dams and locks can be built to control water flow.” 



Table 15. 



Scenario 2 Example Student Responses from School 3 by Theme 



Theme 



Example Responses 



Wetlands 



Water diversion around 
city (often with reuse) 



• “Plant a lot of plants that will hold off some water as 
it floods. They would prevent soil erosion increasing 
safety from water.” 

• “Problem: Not enough wetlands to absorb the rising 
waters. Solution: Promote the growth of wetlands, 
destroy dams, etc.” 

• “Flooding is a key part in forest ecosystems and a 
dam would hurt the agriculture industries. However 
creating a wetlands further upstream will create a 
new ecosystem, purifying the water and slow the 
water down so that it floods at a more constant pace 
instead of seasonal flooding. This will help the 
agriculture down river as the farmers can use the 
river water to irrigate their crops.” 

• “Flooding can be a real problem if the proper 
precautions are not taken. In the case of the farm, 
they could use the flooded water in a reservoir in 
order to help irrigate the farm. Homes could use this 
water (if controllable) to water their lawns and wash 
clothes/dishes/themselves (if clean).” 

• “Flooding is good for the eco-system just like 




Moving Homes 



Floodwalls 
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periodic forest fires are good. So one could use river 
run off, by having vertical popes in the ground that 
would collect water and divert it towards farms or 
have it purified and reused in our water.” 

• “People can’t stop flooding from happening. 
Flooding occurs because people built homes and 
buildings on wetlands where it’s natural for flooding 
to happen. However, if we can minimize the impact 
of the floods so that people are not severely affected 
by them, it will improve many of their concerns. 
When flooding occurs it will take over the houses, 
businesses and farms so we can build a big pipe 
under ground that is usually empty but when floods 
occur, opens and lead the water to and aquifer. But 
before the water enters the aquifer there should be 
an extensive filter system that will clean the water.” 

• “Problem: Houses are too near the river. Solution: 
Move the houses further away from the river.” 

• “I would also suggest moving homes and businesses 
to higher ground so as not to be damaged by any 
flooding that may occur.” 

• “Trying to maintain housing along the river is 
unadvisable because buildings will promote erosion 
and suffer water damages.” 

• “You could build a wall that is strong enough that it 
could block the water from getting to the houses. 
Having a wall along each side of the river allows the 
river to grow higher and wider. The flood water 
would just flow with the rest of the river and won’t 
destroy houses/businesses.” 

• “Recently a new flood protection system has been 
built in London to prevent flooding. If a threat 
exists, a river-wide wall is lowered into place, 
preventing surges and damage. Provided the river 
isn’t freakishly wide, this strategy would work well” 

• “During safe seasons the river banks have short 
walls with periodic pillars, they create a small wall 
but nothing that would destroy the look of the 
country and become annoying.” But when the threat 
of flood occurs the pillars open on either side 
revealing tall metal interlocking plates which when 
extended run along tracks to the next pillar where 
they are fastened to create a water tight wall holding 
back the water.” 
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One example of a problem which was mentioned almost exclusively by students 
at School 2 was that rivers will inevitably flood but that the water did not drain away 
quickly enough. In those cases students proposed solving the problem either with 
supplemental drainage systems or sump pumps. An additional characteristic of the 
responses from School 2, a sample of which can be seen in Table 16, was that they 
tended to focus on problems for individual homes and businesses rather than on the 
overall situation. On the other hand, it was mostly students from School 3 who 
suggested looking upstream (both literally and figuratively) for the source of the issue 
with potential environmental issues exacerbating the amount or frequency of flooding. 
Specifically, many of these students pointed to loss of wetlands as a contributing factor 
in the severity and frequency of flooding. 

Table 16. 

Scenario 2 Example Student Responses from School 2 by Theme 

Theme Example Responses 

Allow river channel to • “Dam part of the river for a while and dig the other 

hold more water part deeper to decrease water height. Add more dirt 

(floodwalls, dikes, to banks to decrease chance of water going over top” 

increase depth) • For the flooding problems you should have the city 

or the town look at the river & do some surveying 
on what part of the river should be built up to make 
a higher bank for less flooding.” 

• “First I would try to build up the river bank so it 
would take a lot more water to over flow.” 

Dams • “You should maybe think about installing a dam 

system so you can control the height of the river 
when needed.” 

• “I would have the city make a dame farther up the 
river.” 



• “Build a dam” 




Move floodwater out 
quickly (sump pumps and 
drainage systems) 



Build in better locations 
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• “A man-made drainage system with a water run-off 
path would also be helpful to keep the water from 
contacting places where its not wanted.” 

• “Add drain tile to all the homes to direct water away 
from the homes . . .put a larger sump pump in all the 
homes. . .put hidden drains in from of businesses so 
water can drain before it enters the building.” 

• “Use drainage and tileing pipe to help the water flow 
to somewhere else away from the homes.” 

• “Build the house up higher like a hill. . .Don’t build 
the house by the water.” 

• “Build homes on higher ground and if they can’t be 
built higher just put a levi or a wall/dam that would 
have water going around the homes, businesses, 
etc.” 

• “Don’t live by water” 



Although many female and male students suggested a lack of containment 
structures including dams, dikes, levees and floodwalls and buildings built too low or 
close to the river as problems in this scenario differences also existed. Many males 
discussed temporary solutions to the flooding issue such as sandbags and portable 
floodwalls the solutions proposed by females tended to be more permanent. Another 
problem category unique to females was the underutilization of excess water. Females, 
much more so than males, were interested in finding ways to put the excess water to 
work such as in irrigating fields during dry spells, filtering and storing in underground 
aquifers or for providing an alternative energy source. On the other hand males were 
much more likely to discuss drainage and wetlands problems. Example responses by 
gender can be seen in Table 17 and Table 18. 




Table 17. 
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Scenario 2 Example Student Responses from Females by Theme 



Theme 



Example Responses 



Find other beneficial uses 
for the extra water 



Build on higher ground 
or raise buildings 



Dams, dikes and levies 



• “Since the farmers further from the river could use 
water, streams could be made coming off the river to 
bring water to the farms and lessen the water in the 
river. This would be the first choice if possible.” 

• “Build an artificial river (canal directing the 
flooding to specific points where the water could be 
used efficiently like farming areas).” 

• “When building new buildings, build them a safe 
distance away from the river.” 

• “I would suggest moving homes and businesses to 
higher ground so as not to be damages by any 
flooding that may occur, if homes cannot be moved I 
would make the banks of the river higher so that less 
flooding would occur.” 

• “Since there must be a long-term solution for the 
potential flooding of this river, simply adding 
sandbags would not be optimal. . .There is a spot 
about 10 miles upstream that would be a prime spot 
to build a dam. There was a beaver dam up until a 
few years ago here, but it has since been destroyed 
and the beavers have left, which I believe is the 
reason that there is now an issue with flooding that 
was no there before.” 

• “Build dams to block water when its overflowing.” 



Table 18. 



Scenario 2 Example Student Responses from Males by Theme 



Theme 



Example Responses 



Temporary solutions 

(sandbags/portable 

floodwalls) 



Dams, dikes and levies 



• “The first way I would fix it is by putting sand bags 
along the river to keep it from going over.” 

• “I would build the banks up on the river so it would 
be harder for it to flood the homes, businesses and 
farms. Before that I would some water pumps to 
pump the water. Once the water was gone I would 
clean the dirt up.” 

• “One this that could be done is the construction of 
levies along the river bank. These will prevent flood 
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levels from reaching buildings or farms.” 

• “A small dam with a reservoir upstream of the town 
would help in that the flow of water would be 
regulated and could be controlled. However, dams 
can cause problems with the natural ecosystems, so 
this problem would have to be addressed according 
to the location of the dam.” 

Drainage systems and • “Excavate a channel from below the town & 

rerouting reconnect it to the river downstream. The channel 

would be lined with concrete so it wouldn’t erode. 
The channel will get lower as it approaches the river 
so water will drain toward the connection down 
stream.” 

• “Put wide, vertical pipes in the ground surrounding 
the river. Have the pipes go a good distance into the 
ground, and have many small pipes branching off 
the main pipe going deeper into the ground (like the 
roots of a tree). This would allow the water to be 
absorbed deeper in the ground which would allow 
for more water to be absorbed, faster, which would 
lessen the effects of flooding, while keeping the soil 
moist to prevent any droughts or other problems due 
to lack of water.” 

Upstream fixes (i.e. • “Plant a lot of plants that will hold off some water as 

wetlands) it floods. They would prevent soil erosion increasing 

safety from water.” 

• “Plant trees along the river to take in more water” 



A multivariate analysis of variance showed no statistically significant 
differences in the four traits of creativity measured by the judges at the .05 alpha level 
for school, gender or a school/gender interaction. In addition to the MANOVA 
procedure the data was analyzed using multiple regression to explore differences in 
each of the four traits based on gender as well as the number of engineering, advanced 
science and traditional technology education classes reported by each student. The 
results of these multiple regression analyses for each of the four traits with significant 
coefficients at the alpha level of .05 are presented in Table 20, Table 21, Table 22 and 
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Table 23. The multiple regression analysis for originality was not significant at the .05 
level. The average judge scores on measures of creativity by school and gender are 
presented in Table 19. 

Table 19. 



Scenario 2 Average Scores 





Flexibility 


Fluency 


Originality 


Elaborateness 


School 1 


2.333 


2.381 


2.463 


2.219 


School 2 


1.970 


2.010 


2.103 


1.313 


School 3 


1.944 


2.083 


2.344 


2.022 


All Males 


1.996 


2.063 


2.201 


1.749 


All Females 


2.222 


2.356 


2.686 


2.130 



Table 20. 



Scenario 2 Fluency 





Unstandardized 

Coefficients 


Standardized 

Coefficients 


Sig. 




B 


SEB 


P 




Constant 


1.328 


.243 




.000 


Gender 


0.155 


.340 


.051 


.649 


Engineering 

Classes 


0.298 


.172 


.187 


.086 




Advanced 
Science Classes 



0.244 .084 



.384 



.005 
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Technology 

Education 0.332 .139 .312 

Classes 

Note. .151 



.019 




Table 21. 
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Scenario 2 Flexibility 





Unstandardized 


Standardized 






Coefficients 


Coefficients 


Sig. 




B 


SEB 


P 




Constant 


1.396 


.218 




.000 


Gender 


0.110 


.304 


.041 


.718 


Engineering 

Classes 


0.294 


.154 


.208 


.059 


Advanced 
Science Classes 


0.200 


.076 


.354 


.010 


Technology 

Education 

Classes 


0.250 


.125 


.264 


.048 


Note. R^= .136 




Table 22. 
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Scenario 2 Originality 





Unstandardized 


Standardized 






Coefficients 


Coefficients 


Sig. 




B 


SEB 


P 




Constant 


1.830 


.191 




.000 


Gender 


0.285 


.268 


.119 


.290 


Engineering 

Classes 


0.186 


.135 


.150 


.173 


Advanced 
Science Classes 


0.168 


.066 


.339 


.013 


Technology 

Education 

Classes 


0.104 


.110 


.126 


.345 


Note. R^= .126 




Table 23. 
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Scenario 2 Elaborateness 





Unstandardized 


Standardized 






Coefficients 


Coefficients 


Sig. 




B 


SEB 


P 




Constant 


1.739 


.221 




.000 


Gender 


0.098 


.309 


.035 


.752 


Engineering 

Classes 


0.338 


.156 


.235 


.033 


Advanced 
Science Classes 


0.094 


.077 


.163 


.226 


Technology 

Education 

Classes 


-0.230 


.127 


-.239 


.073 


Note. R^= .135 



Summary of results 

Several interesting observations can be made about the nature and creativity of 
problems identified in these two scenarios by students at the three participating high 
schools. First, differences are much more obvious in the qualitative examination of the 
data than in the quantitative measures of creativity. In fact, the most consistent predictor 
of creative measures across both scenarios was the number of advanced science classes 
taken by a student. 

Secondly, the qualitative analysis of response categories shows some striking 



differences in the responses of students, particularly in the first scenario which had an 




86 

overall greater number of response categories identified. The types of problems 
identified by school 1 (with engineering courses) and school 3 (with advanced science 
courses) were much more similar than the types of problems identified by students from 
school 2 (with traditional technology education courses). In addition, the categories and 
responses from school 1 and 2 took a much broader approach to the problem, that of an 
outside engineer, where just about anything was on the table from a complete review of 
the home design to advanced power saving technology and alternative energy schemes. 
In contrast the responses from school 2 took a much narrower view of engineering, 
frequently choosing to look at simple changes a homeowner could make themselves 
such as replacing light bulbs, appliances and windows. Overall it looks as if there are 
indeed differences in student responses by school, gender and coursework though these 



differences manifest themselves in different ways. 




Chapter V 
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Summary and Discussion 

Summary 

The primary purpose of this study was to examine the state of engineering 
problem finding ability in high school students at three Minnesota high schools. This 
goal was undertaken through an examination of the state of engineering problem finding 
amongst students and schools with a variety of backgrounds and a search for differences 
between groups of students. The investigation of differences took two distinct paths 
which both relied on the same set of student responses to two engineering scenarios. 

The first path explored the differences in problem finding ability through the use of 
statistical analyses of creativity measures scored by a panel of judges. The second path 
took a more qualitative approach exploring the different problem categories and ideas 
generated by students in different groups. Students were presented with two different 
scenarios which might be faced by an engineer and asked to generate as many problems 
as possible which, if solved, would improve the situation. Students were asked to be as 
clear and thorough as possible and to apply their knowledge and skills to generating 
problems. 

As has been described in the preceding chapter there were, indeed, differences 
found amongst the various student responses. The key differences which were identified 
are summarized below using the guiding research questions as a framework for 
discussion of the results. 

Differences by school 

Although the MANOVA procedure identified no statistically significant 
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differences in the measures of creativity by school a qualitative analysis of categories in 



student responses did indicate some differences. Specifically, students from School 2 
tended to think about things on a smaller scale while students at School 1 and School 3 
tended to look at the big picture. For example, within the energy efficiency scenario this 
manifested itself as things that an individual homeowner could do to improve efficiency 
versus major changes in the design and construction of homes which were more likely 
to come from students in School 1 and School 3. 

In addition, students from School 2 tended to write shorter and less thorough 
descriptions of the problems they identified than students in the other two schools. They 
were also more likely to identify no problems at all or explicitly write that they couldn’t 
think of anything. An informal discussion of the data with one of the judges who had 
just completed the scoring of the data indicated similar observations. 

School 2 had different responses. They were usually shorter. At first I thought 
that they were just not given as much time as the other students, but as I kept 
working I believe they were really trying, but just didn't have the knowledge to 
talk intelligently on the subjects. That doesn't go for all of them. Many had 
good things to say. But many of them just wrote a sentence or two or simply 
said they couldn’t think of anything to write down. . . There was definitely a 
general lack of a problem solving mentality amongst many of the school 2 
students. Granted, there were several with excellent ideas, but these did not 
occur as often as the other schools. 
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Another interesting observation was that although students were not prompted in any 
way to make drawings or sketches some of the students from both School 1 and School 
3 did so anyway. None of the students from School 2 made any sketches or drawings. 
Differences by gender 

Again, the MANOVA procedure identified no statistically significant 
differences on measures of creativity by gender. There were two instances in the 
multiple regression analysis of the first scenario where gender was identified as 
statistically significant, but these are suspect as well because of the very small number 
of female students. There were some differences evident in the qualitative data. 
Specifically, in the first scenario the females were more likely to mention changing the 
design or layout of the house and to suggest that appliances, electronics and lights left 
on were significant contributors to inefficiency. In the flooding scenario the females 
mentioned only problems solved by permanent solutions such as dams and diversion 
systems as well as showing a significant interest in finding ways to take advantage of 
the extra water. 

Overall the differences by gender were not as pronounced as might be expected. 
This may have been confounded by the much smaller number of female students and 
the much stronger science background of the females. Still, hints of differences did 
seem to exist. This was more evident in the types and nature of problems identified than 
in the creativity of them. 

Differences by coursework 

Because coursework was a continuous rather than categorical variable it could 
not be evaluated qualitatively in the way that gender and school were. Instead, 
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coursework in engineering, advanced science, and technology education classes was 
used in a multiple regression analysis to discern any differences. This was an area 
where differences were clearly evident. 

In both scenarios students with more advanced science classes scored higher on 
measures of both flexibility and fluency. Interestingly, the fluency scores were also 
significantly influenced by both engineering (in the first scenario) and technology 
education (in the second scenario). The engineering and technology education classes 
seemed to have a larger effect, measured by coefficient size, than the advanced science 
classes but they were not always significant. None of the classes was significant for 
measures of originality in both scenarios though there was a slight effect of advanced 
science classes on originality in the second scenario. Finally, in both scenarios students 
with more engineering classes scored higher on elaborateness. One additional 
interesting observation was that technology education students had a statistically 
significant decrease in elaborateness scores for the first scenario. 

Discussion 

Based on the results of this investigation it is clear that engineering problem 
finding is not a simple task, nor is it one with which most students are comfortable. 
Despite directions for both scenarios clearly indicating that the task was to find 
engineering problems and the title of the instrument being an Engineering Problem 
Finding test almost all of the students responded with engineering solutions. This was 
not the case universally but, even among students who gave problems, something 
compelled most of them to also give solutions to those problems. One potential 




91 

explanation of this might be the de-emphasis of finding problems within some models 
of problem solving. 

While this study is unique among high school programs there are signs of 
interest in researching in this vein among collegiate engineering education researchers. 
A study published by Atman, Yasuhara, Adams, Barker, Turns and Rhone (2008) as 
this dissertation was under development explored the breadth of problem scoping for 
both freshman and senior engineering students. Much as this study indicated some 
differences in the problem finding of students from different school and coursework 
backgrounds the Atman, et al. study found differences between freshman and senior 
engineering students. 

One of the areas where results seem to align is in the substantialness of student 
responses. Atman, et al. (2008) noted that seniors gave significantly more substantial 
responses to their scenario than freshman. One hypothesis for explaining this 
discrepancy would be that seniors have both more foundational knowledge and a better 
understanding of the engineering design process than freshman. Looking at the data 
from this study it was clear that the responses from schools 1 and 3 were frequently 
more substantial than the responses from school 2. In fact, judges noted this in their 
comments. Remember that students in school 1 all had some engineering coursework 
and students in school 3 rigorous physics coursework while students in school 2 were 
from traditional technology education classes. Given the results of the Atman, et al. 
(2008) study this would seem to indicate that the substantialness of the student response 
would be greater among students more familiar or experienced with engineering 



principles. 
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In addition to noting differences in how generally substantial the responses were 
Atman, et al. (2008) also noted that most seniors discussed a wider variety of factors 
than their freshmen counterparts. Similarly, the students from schools 1 and 3 with 
more engineering and science background tended to look at a wider variety of problems 
than students with a technology education background from school 2. 

Another issue which came up was that several students either flat out stated they 
had no ideas or left their response sheets entirely blank for one or both of the scenarios. 
A hint about why this might be was found on several other student responses where 
students had specifically written that they found the scenarios “too vague”. This 
anecdotally confirms the concerns of engineering researchers such as Jonassen, Strobel, 
& Lee (2006) and Richard Felder (1987) who are concerned that engineering students, 
schools and classes are too frequently working with well- structured problems. This is 
problematic when engineers reach the real world and are faced with ill-defined 
situations. 

This inability to deal with ill- structured problems is a potential concern for 
engineering educators as researchers such as Atman, et al. (2008) have specifically 
noted the importance of engineers’ ability to define problems in addition to solving 
them and a tolerance for ambiguity. It further contrasts with the work of Dorst & Cross 
(2001) where the most creative engineers, a desirable trait in their study, were those 
who challenged the bounds of the problem and looked at bigger picture problems and 
solutions. Dorst & Cross (2001) specifically suggest that creative engineers like to 
“manipulate assignments, because they are often too narrow” (p. 432). If these are the 
goals and desired traits of engineers there seems to be some work left to be done at the 
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high school engineering level where students remain frustrated, rather than motivated, 
by ill- structured problems. 

As far as differences among the schools, genders and coursework histories go, it 
is clear that there are some differences. This is especially true in the most common 
types of problems identified by the various schools and genders and somewhat less so 
for the four measures of creativity. Of particular note was that students exposed to 
engineering and advanced science courses were more likely to look at the big picture 
and think though the scenario as a consulting engineer might while students exposed to 
a traditional technology education environment were more likely to discuss things on 
smaller scale. 

Limitations 

Some care must be taken when interpreting the findings of this study. First, this 
study was limited by the schools and students which were available for participation. 
There are many other factors which might have come into play and which could not be 
controlled as neither the schools nor students were randomly selected. This means that 
the findings here cannot be generalized to larger populations but should instead be 
indicators of areas for future research and interest. Second, it is important to understand 
the overarching goal of this study was to examine and describe the state of engineering 
problem finding in high school students. Its purpose was not to suggest, imply or 
determine whether one curriculum or style of education led to superior problem finding 
skills. The primary purpose was to provide descriptive evidence to aid in the 
development and interest of future research in the area of engineering problem finding. 
Implications and Recommendations 
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The appropriate definition of a problem can mean the difference between 
success and failure of a project or a career. If students are to be successful they must 
develop a firm grasp on the ability to discern problems worth solving. While some 
students tested in this study were able to identify problems in each of the two scenarios 
neither the quantity nor scope of problems identified was particularly impressive. The 
unfamiliarity with finding problems in ill-structured scenarios should be an important 
wake up call to educators at large and particularly to the engineering education 
community. Engineering and design is more than drawing up plans for something, it is 
also determining what to draw plans for in the first place. The understanding of this, 
which is now becoming clear in collegiate engineering education literature, is still far 
from adequate among the newer field of high school engineering education. 

Perhaps then, one of the most significant contributions of this study is the 
identification of the continued difficulty that students have in dealing with ill-defined 
problems. If students, regardless of whether they are bound to be engineers, are to be 
successful in the real world they must learn to deal with situations which do not have a 
single correct answer and which cannot simply be solved through the application of 
technical or mathematical skill. This situation itself is not one which should be difficult 
to address. Indeed, several students showed much promise in the realm of engineering 
problem finding so the skill can exist among high school students. Still, the large 
majority of high school students seem to need significant assistance in developing the 
critical skills of dealing with open-ended, ill-defined problems and engineering problem 



finding. Perhaps the issue is one of awareness about the importance of problem finding 
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ability among high school educators and the requisite pedagogical content knowledge to 
teach this skill. 

Additional emphasis on problem finding in engineering could and should be 
included in technology and engineering instructor professional development as well as 
collegiate coursework. As teachers become more familiar themselves with the role and 
work of engineers and develop methods for teaching real-world engineering skills there 
should be some improvement in the abilities of their students to find engineering 
problems. 

It is hopeful that this study spawns additional interest and research on the ability 
to teach technological problem finding and problem solving though engineering. 
Although we understand problem finding and problem solving to be critical aspects of 
engineering and design we are still in the infancy of understanding the best methods for 
teaching these skills, especially at the high school level. Beyond teaching, we must also 
be able to assess the technological or engineering problem finding ability of students for 
both formative and summative purposes. Atman, et al. (2008) also suggests that looking 
at problem scoping might be a tool for assessing and improving design education at the 
collegiate level. 

One of the many areas of future inquiry suggested by this study is the question 
of why differences exist. What in the curriculums of school 1 and school 3 might have 
made students respond differently than the students in school 2, both in terms of the 
types and creativity of problems identified? If we determine that the ability to find 
engineering problems, creative engineering problems, within a scenario is a desirable 




96 

trait how can this be taught and integrated effectively into the variety of engineering and 
technology education programs? 

While this study focused on the development of creative problems by high 
school students exposed to an engineering scenario it would be useful to know 
something about how those problems develop into designs and finally actual solutions. 
Clearly, the development of solutions is closely tied to the development of problems. 
Within this study that is evidenced by the great number of students compelled to list not 
only the problems they identified but what they would propose to do about them. While 
the development of engineering designs has been the study of collegiate engineering 
education researchers it has not really been looked at for the high school level despite 
the increasing level of interest in high school engineering programs. We now know that 
some differences exist in the problems identified by high school students given an 
engineering scenario but a logical next step would be to see how these problems 
become designs and then solutions. 

Of course, one key concern for technology and engineering educators is 
ensuring that students are, and stay, interested in their programs. This is especially a 
concern at the high school level where many elective courses compete for students. A 
potential avenue for this suggest by Atman, et al. (2008) is making sure recruitment 
materials and introductory courses make clear the global and social concerns of 
engineering rather than focusing too narrowly on technical engineering concepts and 
problems. As evidenced by the results of this study engineering scenarios can generate a 
wide variety of problems which are not all of a technical nature. 
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If engineering hopes to have an enduring place in our high school curriculum it 



must provide an advantage for all students who take the classes, regardless of whether 
they go on to be engineers. Engineering, by its very nature, is reliant on problem finding 
and problem solving skills. Thus, it is in a unique place to teach students these skills, 
which are continually discussed as critical for all students, in an authentic environment 
which promotes the transfer of learning. 
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This test of problem finding ability is part of a research study being conducted by a researcher at the 
University of Minnesota studying the problem finding ability of high school students. You should 
already have been provided with an informational sheet detailing this study. It is important to note that 
it is up to you to decide to participate in this anonymous study and there is no penalty for failing to 
participate but in the interest of fairness all students will be asked to complete this exercise in problem 
finding. Should you choose not to participate in the study your paper will not by looked at further by 
the researcher. Please indicate that you have been informed of the purpose and nature of this study by 
checking the appropriate line below. 

□ YES, I agree to participate in this research study and have received sufficient information about 
the purpose and nature of the study. 

□ NO, Please do not use my answers as part of this research 

Demographic Information 

Please indicate whether you are a □ MALE or □ FEMALE 



Please list the names of any industrial technology, technology education, engineering or AP or enriched 
science courses you have taken in the past as well as what grade you were in at the time: 



Course Name 


Grade 











































Die following pages contain two scenarios which might be encountered by an engineer. Please read 
through each scenario and think of how you would go about improving each situation. Y ou should 
rouglily divide the remaining time in half and spend about half the time working on each scenario. Y ou 
may use the remainmg pages to respond to the scenario. If you use any of your own paper please turn 
that in with the test booklet. 

1 1 



no 



Scenario 1: 

Y ou have been hired as a consulting engineer for a residential architectural and construction firm which is 
concerned that the houses it is designmg are not as energy efficient as possible. With the recent rise m en- 
ergy costs they are concerned that high energy use will discourage potential home buyers. 

Guidelines: 

1) Find original and novel problems which if solved would increase the energy efficiency of homes 

2) Write down and describe the problems you would solve in a way that other people can understand them 
as clearly as possible 

3) Apply your knowledge as much as possible 



I 1 



Ill 



Scenario 2: 

Y ou have been hired as a consulting engineer by a government organization which is concerned about 
flooding along a local river and it’s impact on homes, businesses, and faints. 

Guidelines: 

1) Fmd original and novel problems which if solved would improve tlie flooding situation 

2) Write down and describe the problems you would solve in a way that other people can understand them 
as clearly as possible 

3) Apply your knowledge as much as possible 



I 1 
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University of Minnesota 



Twin Cities Campus 



Business and Industry Education 



1954 Buford Avenue, Room 425 
St Paul. MN55108-6197 



Department of Work and Human Resource 
Education 

College of Education and Human 
Development 



Office: 612-624-3004 
Fax: 612-624-4720 
www.cehd. umn. edu/whre 



December 3, 2008 

Dear Parents and Guardians: 

Your child is invited to participate in a research study conducted by Benjamin Franske, from the 
University of Minnesota Department of Work and Human Resource Education. As part of my Ph.D. 
dissertation I hope to iearn more about how students find engineering probiems. Your chiid's school and 
teacher have agreed to participate in this study and your child's class was selected because of a focus on 
engineering. 

To be fair aii students in your chiid's ciass wiii be asked to compiete an anonymous written probiem 
finding test. This test wiii take about 55 minutes during their reguiar class period. The test wiii consist of 
a short demographic section and two scenarios which your chiid wiii be asked to evaiuate for 
engineering probiems. Aithough ail students will complete the test only tests from students who have 
agreed to participate in the research study wiii be returned to me. If you do not wish to have your 
student's test used in this research study you may either sign and return the form at the bottom of this 
letter or ask your child to check the box on the test indicating that they do not wish for it to be used for 
research purposes. 

Participation in this research is voiuntary, anonymous and invoives no risks beyond those normaiiy 
encountered in the schooi environment. This test wiii not be graded or scored by your chiid's teacher 
and wiii not affect their grade in any way regardless of participation. The primary benefit of participation 
for your chiid is additionai practice in technoiogicai and criticai thinking skiiis. Additionaiiy, it is expected 
that this research wiii improve our understanding of technoiogy and engineering education. However, i 
cannot guarantee that you or your chiid wiii personally receive any benefits from this research. 

Your chiid's participation in this research is entireiy voluntary. Your decision whether or not to iet your 
child participate wiii not affect your relationship with their teacher, school or the University of 
Minnesota, if you decide to aiiow your child to participate, you are free to withdraw your consent and 
discontinue your chiid's participation at any time without penaity. 

In accordance with federal law this research project and copies of the probiem finding test are reviewed 
by the University of Minnesota institutionai Review Board prior to test administration, in addition, your 
chiid's teacher wiii have extra copies of the test which students may bring home after the test is given if 
you are interested in a copy. Finaiiy, if you are interested in the final report of this research it wiii be 
avaiiabie from your schooi principai or directiy from the researcher once the project is compieted. 

If you have any questions about the study or your child’s participation piease feei free to contact me via 
email at frans046@umn.edu or by phone at 952-200-8945. You are also welcome to contact my advisor 
Dr. Theodore Lewis at 612-624-4707 or via emaii at iewis007@umn.edu . If you have questions 
regarding your chiid's rights as a research subject or wish to speak with someone other than the 
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researcher, contact the University of Minnesota Research Subjects' Advocate Line, D528 Mayo, 420 
Delaware St, Southeast, Minneapolis, Minnesota 55455; (612) 625-1650. This Office oversees the review 
of the research to protect your rights and is not involved with this study. 



Sincerely, 




Ben Franske 
Researcher 



If you do not give your consent for your child's participation in this study of probiem finding, please 
sign the bottom portion of this form and return It to your child's teacher by December 10 or ask your 
child to indicate that their test should not be used for research on the test itself. 

I DO NOT give consent for my child (name) to participate in this study 

Print Parent/Legal Guardian name: 

Parent/Legal Guardian Signature: Date 



UNIVERSITY OF MINNESOTA 
STUDY INFORMATION SHEET 

An analysis of high school engineering problem finding ability 

Y ou are invited to participate in a research study about engineering problem finding ability in high 
school students. You were selected as a possible participant because your school and teacher have 
agreed to participate. Y ou should already have received a letter to your parents detailing your potential 
participation in this voluntary and anonymous study. It is up to you and your parents to decide if your 
anonymous answers will be used as part of this research but in the interest of fairness all students in 
your class will be completing the in-class test. 

This study is being conducted by: Benjamin Franske from the Work & Human Resource Education 
department at the University of Minnesota. 

Background Information 

The purpose of this study is to examine the ability of high school students to find and define 
engineering problems in real world scenarios. I am interested primarily in describing what types of 
problems high school students find when presented with an engineering challenge and in looking at 
how engineering, science and technology classes prepare you for these challenges. 

What you will be asked to do: 

All students in your class will be taking a problem finding test which will take about 55 minutes to 
complete. It will include some background demographic questions in addition to two scenarios. If you 
agree to be in this study your test booklet will be forwarded on to the researcher, if you decline to 
participate in the study your test booklet will be destroyed and not seen by the researcher. The study is 
voluntary and anonymous; at no time will you be asked to provide identifying information. 

Risks and Benefits of being in the Study 

Participating in this study has no known risks other than those normally encountered in school. 

The benefits to participation are: Additional practice in technological and critical thinking skills. 
Additionally, it is expected that this research will improve our understanding of technology and 
engineering education. However, I cannot guarantee that you will personally receive any benefits from 
this research. 



UMN IRB Code # 0810E49041 
Version Date: 12/3/2008 
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Confidentiality: 

This study will be conducted anonymously; at no time will you be asked to provide identifying 
information. Responses from your test booklet may be used or published in the final report but will not 
be linked to you. 



Voluntary Nature of the Study: 

Participation in this study is voluntary. Y our decision whether or not to participate will not affect your 
current or future relations with your school, teacher or the University of Minnesota. There is no penalty 
for choosing not to participate or withdrawing from the study at any time, it will not affect your grades, 
graduation or college admittance in any way. 

Review of the Study: 

This study has been reviewed and approved by the University of Minnesota Institutional Review Board 
(IRB). The IRB has determined that this study meets the ethical obligations required by federal law 
and University policies. If you have questions or concerns regarding this study please contact the 
Investigator or Advisor. If you have any questions, concerns, or reports regarding your rights as a 
research subject, please contact the IRB Administrator. 

Copies of the Test and Study Report: 

Following administration of the test additional test booklets will be available for you to take with you 
for your records. Your original test booklet camiot and will not be returned to you as it contains no 
identifying information. Copies of the final study report will be available from your school principal or 
directly from the researcher at the conclusion of the study. 

Contacts and Questions: 

The researcher conducting this study is: Ben Franske. You may ask any questions you have now. If 
you have questions later, you are encouraged to contact them at the University of Minnesota, 952- 
200-8945, frans046(g,umn. edu . Questions may also be directed to the study advisor Dr. Theodore 
Lewis at the University of Minnesota, lewis007fa)umn.edu . 

If you have any questions or concerns regarding this study and would like to talk to someone other 
than the researcher(s), you are encouraged to contact die University of Minnesota Research Subjects’ 
Advocate Line, D528 Mayo, 420 Delaware St. Southeast, Minneapolis, Minnesota 55455; (612) 625- 
1650. 

This sheet is for you to keep for your records. 



This project has been reviewed by the University of Minnesota IRB as 
required by the Code of Federal Regulations Title 45 Part 46 
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University of Minnesota 



Twin Cities Campus 



Business and industry Education 



1954 Buford Avenue. Room 425 
St Paul. MN 55108-6197 



Department of Work and Human Resource 
Education 

College of Education and Human 
Development 



Office: 612-624-3004 
Fax: 612-624-4720 
WWW. cehd. umn. edu/whre 



June 2008 
Dear Principal: 

My name is Benjamin Franske. I'm currently a Ph.D. student in Technology and Engineering Education at 
the University of Minnesota. I am conducting a dissertation study on the initial stages of engineering 
problem solving at the ll"' grade level. Students in this study will receive a small set of real world 
engineering problems and asked what questions they would ask in order to solve the problem as well as 
essential demographic questions including the nature of any prior courses in science, engineering or 
technology. I am seeking your permission to conduct this research within your school. No student names 
or correspondence with individual students will be required and all collected data will be treated 
confidentially. The study will be approved by the University of Minnesota Institutional Review Board and 
done in conformity with the research regulations of your district. 

Data for this study will be collected in Fall 2008. If your school has technology or engineering courses I 
would like to conduct the survey in those courses, if not I would like to collect data in a general 
education course. Data will be collected from schools with varied income levels, rural, suburban and 
urban schools as well as schools with technology or engineering programs and those without, it is 
expected that participating students will require about one standard (45-55 minute) class period to 
complete their part in this study. 

The resuits of this study will be useful for schools and curriculum designers, particularly in the science, 
technology and engineering fields. Results and a copy of the final report will be available at the 
conclusion of the study. If you agree to allow me to conduct this research in your school I will need a 
letter of cooperation. A sample letter is enclosed; please return your letter on school stationary. Please 
also indicate what, if any, technology education, industrial arts or engineering courses your district has. 
Please send this letter to: 

Ben Franske 
6104 Arbour Ave. 

Edina, MN 55436 

If you have any questions about the study, your school's participation or the letter of cooperation please 
feel free to contact me via email at frans046(S)umn.edu or by phone at 952-200-8945. You are also 
welcome to contact my advisor Dr. Theodore Lewis at 612-624-4707 or via email at Iewis007(aumn.edu . 

I look forward to partnering with you on this important study and thank you for your assistance. 



Sincerely, 





Ben Franske 
Researcher 



Dr. Theodore Lewis 
Advisor 



Driven to Discover" 



Sample Letter of Cooperation 



Must be on school letterhead 

<Date> 

Ben Franske 
6104 Arbour Ave. 

Edina, MN 55436 

To Whom It May Concern: 

Benjamin Franske has requested permission to collect dissertation research data about engineering 
problem solving from students at <name of school/dlstrlct>. 1 have been Informed of the purpose of the 
study as well as the data collection procedures and approximate timeframe. I have also been given an 
opportunity to ask questions of the researcher and advisor. 

1 understand that this research will be carried out following ethical research principals and will be 
approved by the University of Minnesota Institutional Review Board prior to data collection. Individual 
participant involvement in this study is strictly voluntary and no student names or personally identifiable 
information will be collected. I understand that parents/guardians and students will be notified In 
advance about this research and provided an opportunity to decline involvement without penalty if they 
so choose. 

Therefore, as a representative of <name of school/districts, I am authorized to grant permission to have 
the researcher, Benjamin Franske, to recruit participants and collect data at <name of school/dlstrlct>. 

If you have any questions please contact me at <your contact lnformatlon>. 



Sincerely, 



<Name of authorized representatlve> 

<Official title> 

Additional Information: 

Our <school/district> has the following technology education/industrial technology/engineering 
courses: <list any courses you have>. Data collection would be most welcome in <list courses which 
would be available for data collections which would Include roughly <estlmated number of studentss 
students. 
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Reviewer #: 

Dear Judge, 

Thank you for taking the time out of your busy schedule to help with the evaluation and scoring of these 
engineering problem finding tests. Hopefully you'll find this a fairly straightforward process but if at any 
time you are confused or unclear please don't hesitate to contact me ( frans046t5)umn.edu or 952-200- 
8945). Attached you will find several things which should clarify what your task is and how to go about 
it. 

First, I'll include a short background on the study as a whole so you have some idea where this is going 
and what your responses will be used for. Second, I'll describe what I'm looking for from you and some 
basic information about the scoring of creativity. Third, will be some specific instructions for the scoring 
of student responses. Attached will be a sample student response which I have filled out myself to give 
you some ideas about different directions students might go as well as a rubric to further clarify the 
scoring and an example real student response which I have scored and given some rationale for how I 
determined scores in the specific instructions for scoring a scenario. 

I really appreciate your willingness to help out with this project and please do not hesitate to let me 
know if there is anything I can do for you in the future. 

Thanks, 



Ben Franske 
University of Minnesota 

Department of Work & Human Resource Education 

Study Background 

Many of the problems in our modern world which engineers are asked to solve are not well structured. 
For example, an engineer may be asked to improve the energy efficiency of a building, fix a problem 
with the water in a remote village or alleviate a traffic problem in a city. Each of these problems has an 
identifiable goal but none an immediately apparent path to solution. The path chosen by the engineer is, 
for a large part, determined by their identification of a specific problem to address which they believe 
will address the larger issue most effectively. This process of need or problem identification is generally 
accepted to be the first step in the design process. It is obviously a critical step as it sets the tone for the 
forthcoming solution yet it is often not one that is especially well addressed in engineering courses and 
curriculum. 

Little is currently known about the ability of high school students to find engineering problems or how 
this ability is related to participation in technology or engineering education courses. Therefore, it is the 
goal of this study to measure and describe the question posing capabilities of high school students with 
a variety of backgrounds including those which have taken technology education or engineering courses 
and those who have not. Students will be presented with two scenarios which might be encountered by 
a practicing engineer. Question posing skills will be evaluated by a panel of judges comprised of 
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engineers as well as technology educators and experts in creativity on the fluency, flexibility, originality, 
appropriateness and elaborateness of the questions posed by each student after examining the 
scenarios. In addition to these scores the panel will also be looking for patterns and categories in 
student response. 

Gauging Creativity 

While there are many methods which have been proposed for gauging creativity this study will use the 
areas of creativity identified by Guilford, namely fluency, flexibility, originality and elaborateness as well 
as the check of appropriateness to weed out irrelevant responses. Because these will be the primary 
areas of your assessment a brief description of each term is included below. 

The fluency factor recognizes that, all other things being equal, those students who generate the most 
ideas per unit of time are more likely to have ideas of interest and significance than those who generate 
fewer ideas. Therefore, fluency is primarily concerned with the number of ideas generated by the 
student. 

Flexibility involves the ability of the student to generate ideas which lead to many different branches of 
investigation rather than staying in a rut. A more flexible student would look at many very different 
ways of addressing a scenario which they encounter. For example, in the case of needing to cross a river 
a less flexible student might identify the problem as a lack of a bridge and propose several variations of 
bridge design while a more flexible student would propose many alternatives such as a water ferry 
system, a tunnel, diverting the river, a bridge, or an aerial ferry. 

Creative students have novel or original ideas. Although the responses must be acceptable or 
appropriate for the scenario the ability to generate uncommon ideas is a valuable measure of creativity. 
We might also call this the ability to "think outside the box". 

In order to be successful the creative student must also be able to clearly communicate and elaborate 
on their ideas. This elaboration allows for the problem to be redefined and further developed by the 
student as well as to clarify the relevance to the given situation. Elaboration may take many forms 
including diagrams and drawings as well as the written word. 

Remember that the goal for students was not to "solve" the scenario but to generate potential problems 
which, if solved, would improve the situation. In many cases the students do identify solutions rather 
than problems. For the purposes of scoring I would ask you to determine what the underlying potential 
problem they attempted to solve was. I think as you look at my example of a scored response you'll see 
how I did that. 

In general, I don't mind how you go about determining where one problem ends and another begins or 
when responses should be lumped together but I do ask that you be consistent. I would also encourage 
you to browse through several of the responses from each school (the school is the first set of numbers 
in the student ID tags (eg. 01-xx-xx) before you start scoring. This will allow you to get some sense of the 
variety of responses and problems identified by students which is important for accurately scoring how 
original ideas are. 
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Scoring the Scenarios 

Note that it may be helpful to look at the example of a scored student response as you read over this. 

The directions here specifically address my rationale for scoring this example. 

1. Copy the student ID to and mark the scenario number and your reviewer number on the 
evaluation form. The key part of the student I D is the first two pairs of numbers (01-08 in the 
example) with the third one being a page number. It would also be helpful if you made note of 
any trends from one school to the next (the first two numbers, in this case 01) to aid in later 
comparison. 

2. Read through the entire student response to the scenario including any additional pages if they 
used them. As you're reading through decide how many disparate problems the student 
identified and what they are. Student responses vary from clearly delineated lists to paragraph 
format to drawings and sketches. It may be helpful to label the problems in the margin of the 
page as you identify them. This is what I did in the example but this is not required and you will 
not be returning the student responses to me so mark them up (or not) as you see fit. Not all 
judges will identify the same problems in student responses but please be as consistent as 
possible about how you identify when one problem ends and another begins. For example, in 
my sample evaluation I treated air seepage through cracks as a different problem than poor 
door/window seals but you may decide to treat these types of responses as one or as three as 
long as you do so consistently across all students. 

3. Once you have read the entire response of the student give the student an overall rating on 
fluency with assistance from the rubric. In the example I gave a fluency rating of 3 because the 
number of problems the student identified is about what I expect to be the average number of 
problems identified. It may be helpful as you're getting started to browse through a few 
randomly picked responses to get an idea of what to expect. Fluency deals primarily with the 
number of problems found. 

4. Do the same for flexibility. My flexibility rating of 3 was because I saw three large categories 
addressed: radiant heat loss, air leakage, furnace efficiency. The more difference between the 
categories the higher this rating should be. For example, in the sample of my own responses I 
listed problems dealing with appliance efficiency, radiant heat loss, air leakage, high fuel costs, 
the marketing perception of high energy costs, inefficient floorplan, etc. because of the large 
number of different categories I would receive a very high flexibility rating. Flexibility deals 
primarily with the number of broad categories the problems fall into. 

5. Now move into the per problem rating. For each of the problems you identified determine 
whether it is appropriate to the scenario. I expect it will be rare to encounter an inappropriate 
problem but if you feel that a given problem has no relationship at all to the scenario this is 
the place to indicate so. 

6. Continuing to look at each problem individually give a score for originality. In general the fewer 
students who identified the problem the more original it is. In the example I scored I gave the 
first problem I identified (lack of roof insulation) an originality score of 2 because based on what 
I've seen looking through student responses this is a very common one. 



7. In evaluating the elaborateness you are scoring how thoroughly each problem was 
developed/explained by the student. In the example I scored I gave the first problem I 
identified (lack of roof insulation) an elaborateness score of 3 because the student explained 
why the problem is important and material to the scenario. 

8. Finally, give any comments or notes you have about the student's response to this scenario. I'm 
particularly interested in anything interesting/different the student described. In the example I 
scored I noted my skepticism about whether electric furnaces were really more efficient, that a 
major focus was reducing air/heat movement and that alternative fuels (electricity vs. fossil) 
were mentioned. You are not required to make comments on all evaluations but if you see 
something interesting, unusual or creative please make a note of it! 
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S&^Pl^ 

Scenario 1: 

You have been hired as a consulting engineer for a residential architectural and construction firm which is 
concerned that the houses it is designing are not as energy efficient as possible. With the recent rise in en- 
ergy costs they are concerned that high energy use will discourage potential home buyers. 

Guidelines: 

1) Find original and novel problems which if solved would inerease the energy efficiency of homes 

2) Write down and describe the problems you would solve in a way that other people can understand them 
as clearly as possible 

3) Apply your knowledge as much as possible 



' — /j]/ leoJiijr^ <^lou^ oyf oaJ to 



— iToulcl Ic^ io ire^wie -fToM^SiU^ 

/d c/Wctcit.i<5 

— * f/<?t hre^cf cdJaQ i>< of 5-foddl^ 



— tie ty ^jzoftw^ o<f off 

— ^ aeer^d^oe^ co<xt^ e^w od^5 

_ 

lAfols ffOc-t. -be cilofh 

- fio«c cc^ li be ^ yb«p »s 

to J/vPfo^ 

- CM ‘II di<n^e6 ^ Ofer, So onl^ 

(Ai^ t>e kxjfrtA./c°o\^, 

fjb-AC C0*\^ ^ 01 rdF« ^ill 

Of- ctobic ecrf-H^ 

p^Me ^ ^ wVi-HU <wci r<tH<ct ( t i/' 

( 1 
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Scenario 2: 

You have been hired as a consulting engineer by a government organization which is concerned about 
flooding along a local river and it’s impact on homes, businesses, and farms. 

Guidelines: 

1) Find original and novel problems which if solved would improve the flooding situation 

2) Write down and describe the problems you would solve in a way that other people can understand them 
as clearly as possible 

3) Apply your knowledge as much as possible 

— U'l-f- p/' bejM'n lAJilV) 

— ret- 

- e’KfU.MC ztbo\i?_ 

— cidfi^ <K'ficX5^ 

— B*.iSA(!S5 Prczf-«cttsi 

— tJux'iAfy fo l&^cp w=f«/ o**J\r 

" /o<<9e. u/Uef CffcajS -iocoiry wot<Y 

— f^r«c!( /vir^K^ -Piooi 

~TnPt*>^ ap6^<5v»j flocti.f)<Ko "Vo c09,^of6 /»U7/€, 

— /3n'iU a. dcrvi to oof\iro) /i\)a -fi<^ 

B*m Id lAjo4of Clncyyicd^ <?favici 

"" f?eo^ iniojhou-^i) 

GWiait. ,^icj^5 -)o Keep tVeAi flavi^ 

- — pajp/P vAjix.^^/<^5M/^KP 06f A. 

^Jeticlop Pcfi^ ^ <y( gin^ 

-tieoeUv’ 

C 1 
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is not material to improving the could possibly improve the situation 

situation presented in the scenario. presented in the scenario. 
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Student ID; OJ_- 0^-01 



Scenario: (1^2 Reviewer#: ‘f? 



Per Scenario Ratings: 

Please rate the flexibility and fluency skills of this student as evidenced on this scenario as a whole 
using the provided rubric as a guideline. 



Fluency; 


1 2 r^J) 4 5 


Flexibility: 


1 2 4 5 



Per Problem Ratings: 

Please rate the appropriateness, originality and elaborateness of each problem the student described 
in this scenario using the provided rubric as a guideline. In the event a student has identified more 
than 12 problems use a supplemental sheet to continue scoring. 



Problem 


Appropriateness 


Originality 


Elaborateness 


1 




N 


1 
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4 
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3 
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'Sc?nof\0 ^ 






Scenario 1 

€been hired as a consulting engineer for a residential architectural and construction firm which is 
concerned that the houses it is designing are not as energy efficient as possible. With the recent rise in en- 
ergy costs they are concerned that high energy use will discourage potential home buyers. 

Guidelines; — 

1) Find original and novel problems which if solved would increase tt -t f\0 1 ' 

2) Write down and describe the problems you would solve in a way tl ( \ / I “ w O ~ Vr -i- 
as clearly as possible 

3) Apply your knowledge as much as possible 
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Student ID: 1 - 


Scenario: 1 


Reviewer#: 





Per Scenario Ratings; 

Please rate the flexibility and fluency skills of this student as evidenced on this scenario as a whole 
using the provided rubric as a guideline. 



Fluency: 


1 QJ 345 


Flexibility: 


12 3 5 



Per Problem Ratings: 

Please rate the appropriateness, originality and elaborateness of each problem the student described 
in this scenario using the provided rubric as a guideline. In the event a student has identified more 
than 12 problems use a supplemental sheet to continue scoring. 



Problem 


Appropriateness 


Originality 


Elaborateness 
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Scenario 2: > 




You have been hired as a consulting engineer by a government organization which is concerned about 
flooding along a local river and it’s impact on homes, businesses, and farms. 


Guidelines: 




1) Find original and novel problems which if solved would improve tl 

2) Write down and describe the problems you would solve in a way tl 
as clearly as possible 

3) Apply your knowledge as much as possible 


01 - 08-02 










c>^ a]\j if thC -Hfigsi-V f\ood^t )'5 ti^or\t 
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